Lost 10 Years Of Work Thanks To MS

[FlyingsCool]

It's a tragedy, definitely but YOU need to learn from this.

Never, ever, ever, leave backup drives (or any drives not necessary to the task in hand - whether OS House-keeping or deep simming or whatever) - connected, when working "under the hood" with Windows - It WILL find a way of ruining your day - or in your case your decade.

I learned that the hard way about twenty-five years ago when I lost pretty much everything from over five years of graphic design in Windows '95. I have never trusted MS since. And I haven't suffered the catastrophic loss you have as a result.

You have made many posts about hardware/ software, etc - It's a bummer but, quite honestly, you should have known about this.

Luckily, these days - data is retrievable to a much greater extent than it was when I lost my data so it's a pretty straightforward process.

You may want to follow the advice in your signature....

[Jafo]

Ah.....foot-in-mouth disease...

Anyone who has NOT had an issue with Microsoft's [lack of] competence probably uses a Mac.

[Priller]

It's a tragedy, definitely but YOU need to learn from this.

Never, ever, ever, leave backup drives (or any drives not necessary to the task in hand - whether OS House-keeping or deep simming or whatever) - connected, when working "under the hood" with Windows - It WILL find a way of ruining your day - or in your case your decade.

I learned that the hard way about twenty-five years ago when I lost pretty much everything from over five years of graphic design in Windows '95. I have never trusted MS since. And I haven't suffered the catastrophic loss you have as a result.

You have made many posts about hardware/ software, etc - It's a bummer but, quite honestly, you should have known about this.

Luckily, these days - data is retrievable to a much greater extent than it was when I lost my data so it's a pretty straightforward process.

And there I was thinking that AussieMan's comment was bad... I was wrong. Your comment is heartless and above all ignorant. Maybe a wee course in comprehensive reading is in order here.

I wonder, is that a chip on your shoulder, or just a huge piece of dandruff? You sir, are a horrible piece of work.

Priller

[gman5250]

Luckily, these days - data is retrievable to a much greater extent than it was when I lost my data so it's a pretty straightforward process.

I'll refrain from comments on your opinion, except for the "Pretty straightforward process" bit.

Straightforward:

The full scan returned 1,239,000 files in 10,596 file folders.
201,959 files were corrupted or unrecognizable by the software.

A partial list:
3D Studio Max 3,590
3D Studio 353
Photoshop 6,250
Bitmaps 111,968
Targa 400
Zip files 2,521
Cab files 1,697
.bgl 43.320

The 3DS Max files are recovered in a format not recognized by my version of Max. To read the files requires I purchase a different version of Max, once I find out which version the files are saved in

The Photoshop files are recoverable, but require I open each one, determine what it is and save it out to a new file structure.
The bitmaps are a bigger pile of the same.

The Zip files are corrupted, requiring that I re-download every zip I ever purchased or acquired.
The purchase records were lost along with the serial numbers.

Ancillary point:
My research is a total loss. I retain most of it in memory, but when engaged in a debate with someone like yourself who is less acquainted with facts I prefer to be able to point to data. Most of the research I do requires obtaining data before it is purged from the internet. Case in point, a forensic study by Mark Skidmore a Michigan State University PHD in economics and his team of graduate students who documented $21 trillion (above the $22 trillion official US debt) in un-adjusted expenditures by the Department of Defense and HUD in the US. I was able to link to and download the actual financial reports from .gov sources before they were purged three days later. The Skidmore study prompted an official response in the form of a bit of quiet legislation, FASB56 eliminated transparency in DOD spending. That didn't get purged BTW, so I downloaded it to christen my new research databank.

Back to the straightforward part.
To re-assemble my data I would need to cross reference every recovered file to every other recovered file of the 1,239,000 files scanned. Doing the math that's a really big number, probably around the 21 trillion I referenced above. The alternative is to hire a DOD level forensic team to run their recovery algorithms to recover my data intact. That runs about $3,000 an hour, depending on who one hires.

I've spent the better part of the week going through the Photoshop files. Luckily I am retrieving a large percentage of my original, layered work intact. I've done about two thousand files...only four thousand to go. I can also retrieve much of my mesh and bitmap work from my published work, but those will require reverse engineering and much re-building to create my 6 aircraft projects, five scenery projects and scenery library models which number in the thousands. The autogen and vegetation models are another huge project...thousands more.

The rest of the million plus files...well, I'll get to those down the road.

In My Humble Opinion
I find it mildly ironic that a company with the market cap of Microsoft cannot provide a data recovery service for the tens of thousands who have suffered catastrophic loss as a result of their software. The former Chairman, Mr. Gates has topped the 100 Billion mark, along with Mr. Bezos, and dedicates his philanthropic work to farming medical mosquitoes and pumping heavy metals into the atmosphere. He does not maintain a foundation for victims of the monopolistic business practices of his former company. I have however, taken comfort over the course of the week in the fact that I have received no less than five notices from MS congratulating me on the MS bonus points I recently "earned" that will allow me to acquire a viking helmet, or perhaps some other trinket to highlight my status as a loyal Windows "user".

You sir, know precisely nothing of my situation and are in possession of no knowledge that entitles you to tell me what I should or should not learn. Your remark is wholly reprehensible and inappropriate. I attempt to conduct myself as a gentleman on this forum, but I do have my limits.

[Ganter]

You sir, know precisely nothing of my situation and are in possession of no knowledge that entitles you to tell me what I should or should not learn. Your remark is wholly reprehensible and inappropriate. I attempt to conduct myself as a gentleman on this forum, but I do have my limits.

Gordon, my apologies I spoke both out of turn and after a few at the pub last night - never a good time to come to the forums.

I guess what happened was I read the thread and all the anger and rage which has never quite abated from my own data loss all those years ago came flooding to the surface and out.

If I was having a go at anybody it was the young man that was me when I stupidly reformatted that hard drive of mine and lost all the contents of my early graphics work - that was five years work - not ten, but it felt equally momentous at the time.

I apologise again, I was lashing out at myself - not you.
I hope you continue to recover those files and wish you well.

[gman5250]

Gordon, my apologies I spoke both out of turn and after a few at the pub last night - never a good time to come to the forums...I hope you continue to recover those files and wish you well.

Your apology is accepted sir, of course.

Your accounting of the circumstances of your post should be a lesson to all of us that we are indeed human and prone to an occasional Momentary Lapse of Reason...to draw upon one of my favorite poets. I had a room mate when I was in my thirties, just picture Bruce Willis. When we went at it we would empty a construction site, but at the end of the day we would go out to the pub and leave trivial matters behind.

It's a sound philosophy.

[Ganter]

Your apology is accepted sir, of course.

Your accounting of the circumstances of your post should be a lesson to all of us that we are indeed human and prone to an occasional Momentary Lapse of Reason...to draw upon one of my favorite poets. I had a room mate when I was in my thirties, just picture Bruce Willis. When we went at it we would empty a construction site, but at the end of the day we would go out to the pub and leave trivial matters behind.

It's a sound philosophy.

Thanks Gordon,

Much appreciated and humbled.

As I said, best of luck with the retrieval of your hard work.

This has inspired me to dig out that old drive - I've got a big box of HDDs from through the years - and see if I can grab any of that data using today's software.

[cavaricooper]

Gordo-

Just an earth bound misfit I...

Here's hoping that "some" of your incredible work can be salvaged. Do let us know how it goes? I think of you in the same frame as Alex Metzger, Manfred Jahn, Uncle Milty, Bernt Stolle, Jan Visser, Ted TuFun et al. The sheer joy you have provided me as a past pilot is priceless. There's no sensation to compare with this. Suspended animation, a state of bliss...

If at some point contributions would aid in your restoration- I (and many others) stand ready.

Best- C

[TuFun]

Gordon, your practical wisdom is truly inspiring and hope all goes well!

Thank you for not lashing out like others have done.

[gman5250]

Gordo-

Just an earth bound misfit I...

Here's hoping that "some" of your incredible work can be salvaged. Do let us know how it goes? I think of you in the same frame as Alex Metzger, Manfred Jahn, Uncle Milty, Bernt Stolle, Jan Visser, Ted TuFun et al. The sheer joy you have provided me as a past pilot is priceless. There's no sensation to compare with this. Suspended animation, a state of bliss...

If at some point contributions would aid in your restoration- I (and many others) stand ready.

Best- C

You are too generous sir, in your assessment of my work. The above mentioned designers are icons in our community, and well worthy of our gratitude and respect.

BTW, Now I will have the song in my head all day. I had done a video a couple of years ago on the Dak from team MJ/JV, featuring that very song. I could not share it publicly due to copyright policies.

Moving on from the excitement of the posts above, there is always a balance of light and dark in all things. I thought I'd share a shot highlighting the performance on the new system, featuring some of the data that has been returned to me from friends. In my recovery efforts of this week I can happily report that I have recovered all of my Photoshop files containing the master art and skins for the DAK. This shot is over my KMMH scenery...and right out my real world front door.

I have locked frames at 60 for video synchronization, but unchecked the system will cruise along at 150 FPS over this monster scenery in a complex model. The stats speak for themselves.

[cavaricooper]

Absolutely stunning. I for one cannot wait until your scenery is available for purchase. I find two dimensions of simming most appeal to me. The airliners high and fast, and with exposure to SOH- the round engines, warbirds and bipes, low and slow (although sometimes not so slow) with ultra detailed scenery.

All that I have seen of your WIPs have definitely piqued my interest, and I shall await the first release with eager anticipation.

With gratitude for all you have done for us, and all that is to come- C

[gman5250]

This has inspired me to dig out that old drive - I've got a big box of HDDs from through the years - and see if I can grab any of that data using today's software.

From what I can ascertain from the recovery attempts I have done, you have a very good chance of recovering your art, so I would have a go at it. If you were using large format PSB, you won't get those back but PSD files are recovering well with layers intact.

I'm using EaseUS.
https://www.easeus.com/

To be honest, I think the program is a bit of a hype and over rating their estimates of recovery probability, but for bitmaps it seems to work. Hope this helps you recover your art.

[gman5250]

Getting Back on Track

So, let's agree that we will not assign blame nor render opinions about what I should have done from here forward. It doesn't fricking matter...it's done. The point of this discussion, at this stage, is to share my experience in my attempt to recover lost data. That said....


Using the EaseUS recovery software.

As I posted above, the scan returned approximately 1.2 million files from the external drive. About 20% of that data is completely corrupt and un-recoverable. The remaining data is broken into folders identified by file type i.e. Adobe Photoshop, Adobe Illustrator, 3DS Max, bitmap, jpeg, TGA, DDS, Microsoft Word Document etc. From what I can see, basic file formats like PSD, MP4 or word docs recover well, but the naming conventions are lost and converted to numbered lists. It is necessary that each file be opened and saved back out with a new name, preferably the name previously assigned to it...If one can remember the names of hundreds of thousands of files.

The data I recovered consisted of my own development work which I estimate constitutes about 33% of the recovered data. The un-recoverable data actually retains the original file names so I can look at those files and document what is permanently lost. Much of that data is large format PSB Photoshop files containing all of the seasonal terrain tiles for my scenery projects. Each geo-data tile was hand altered to represent the five seasons recognized by FSX and Prepar3D. The loss here represents thousands of hours gathering the geo-data and using Photoshop techniques to alter each tile to represent the seasonal colors and snow accumulation.
Another large segment of the corrupted data is indeed 3DStudio Max files, again representing thousands of hours of development time.
The software is clueless when it comes to GMAX...so those files are wiped out completely.
The rest of the corrupt data falls into various categories listed by file type.

Another third of the data was my research, as I said above I retain that in memory but proofs come in handy when pressed on any issue.
The last third of the data was backup for purchased aircraft, scenery, programs and peripherals. Much of this was in the form of compressed files i.e. zip or rar.

When the recovery software scans the partitions it does not preserve structure.
For example, Prepar3D or FSX installers are a composite of multiple file types. Each of those file types are saved out as the various bitmaps, dds, text, dll, exe and other components that create the composite, but the result is to sort those file types out to separate folders. The recovered data is essentially useless. The same is apparently true for compressed file formats or any other structure that bundles multiple file types.

The resulting 1.2 million files represent all of the data, but broken apart and stored in a generic folder structure. It would take an infinite number of monkeys an infinite amount of time to sort those random files back into their proper structure. To my knowledge, those monkeys are presently busy in DC or pumping the rigging algorithms on Wall Street and the COMEX, so they won't be available to me at any time in the near future. Oops....sorry...my bad.

The net result as it applies to my situation.

I am pleased that I am recovering my original hand painted art and source photography. We have also recovered my wife's archive of specialized photographic art, so we are extremely relieved to have not lost everything.
My modeling remains intact in the form of .mdl, .bgl, .bmp. .dds, .xml and other files in the actual simulators. I also have a recent mirror of my previous system which will contain an archive of installed aircraft and scenery for FSX and Prepared. Some of that data will be useful in extracting mesh and bitmaps for use in re-construction of my models..if and when I decide to move into that process.

I'll go over my future plans in another post here, but for now I'm hopeful that this data may be of use to anyone who has experience a catastrophic data loss such as mine.

I do truly appreciate everyone's help, support and offers of financial energy towards recovering my data. To be truthful, I refuse to invest the donations of my friends here into any recovery effort that holds data hostage to justify ridiculous hourly rates of $1,000 to $3,000 dollars. I have a friend across the street, a former Navy Seal, whose brother does DOD level recovery. The good guy rate was so stupid off the chart I nearly laughed in the guys face, but considering his skills...I left that one alone.

For the moment I need to re-evaluate my goals and quite honestly, mastering traditional Japanese wood joinery looks more appealing and the resulting work tends to be of a more permanent nature.

[Jafo]

Unfortunately the exorbitant rates charged for data recovery is simply a mechanism to dissuade people who might have lost photos of Fluffy the cat from 'wasting their time', rather than some black art skillz that justifies the fee.

Really a demand-led fee structure. Your loss could be measured in the 'millions' so they are all going to think 'charge tens of thousands....' ...

[heywooood]

exactly - it's an example of capitalism based on the RANSOM scale..not even remotely based on estimated hours on the job or tech pay scale at all.

they want 'salvage pay' meaning a percentage of the material's net worth - estimated to the highest round ethereal number naturally

[FlyingsCool]

And, to us, the value of your work is worth that. Please don't feel bad about spending money I could give you on recovering your data. It's money you should have gotten in the first place but you generously gave away...
I'm sure you wouldn't be surprised how quickly our donations could add up.

[gman5250]

...they want 'salvage pay' meaning a percentage of the material's net worth - estimated to the highest round ethereal number naturally

If that were the case I could cover it for a couple of hundred bucks.
These guys have obviously never actually sold a product in our little cove in the capitalist ocean. Most products intended for our market net a few tens of thousands on average. Some do quite well of course, but on average the Japanese joinery is probably more lucrative.

[heywooood]

I think it's understood that it means more to YOU than the open marketplace...but in this carve-out, you are the market...and it has ethereal value represented in the time and energy spent by you to create and catalog it-

[Priller]

People,

Let's give Gordon a bit of time to get over this, shall we?

I have the honour to be able to call Gordon my friend for quite a few years now. I can't tell you people what this "mishap" has done to him and his loved ones, but it is shattering.

Let's give him the time to get back on his feet and find some balance in his life, shall we?

What happens next is his choice and his alone. Give the man some breathing space!

Priller

[gman5250]

Brief Update:

I've been sifting through the debris field that was my storage, and doing the cost benefit analysis regarding retrieving the data using a forensics specialist. Bottom line is that the expected revenue from the six retail packages that were basically ready for market would not justify the many thousands of dollars it would cost to retrieve the data. It would basically be a wash.

I've been going through my email records of all my purchases. Many are obsolete, no longer supported or dead links. Many more will require that I re-purchase newer versions of the software. That will be quite a few hundred dollars.
The viable links will take many weeks or months to chase down and re-download.

Regarding FS Pilotshop.

I am appalled that this company has undertaken a policy to de-activate my entire purchase history (including recent), requiring a $4.95 fee to re-instate links to each of nearly eighty purchases. That is nearly $400.00 to access products that I own a legitimate license for, the records of which still exist on the FS Pilotshop website. I will pay those fees under protest, and shall never do business with these crooks again. Sorry if I ruffle feathers here, I call it as I see it...deal with it.

As for the rest of the data, I will need to replace the external storage device and one spinning drive. The damaged storage and existing 5TB spinner holding the "retrieved" data will be retired, awaiting analysis at a later time. In the meantime, I need to have usable storage space.

Regarding Future Projects

A good friend asked me the other day on Skype, what was I planning for my next project.

Friends, please understand that there will be no future projects. This product line was timed to release around the time P3D went to full PBR, but that plan has been eviscerated with the destruction of the assets.
Building a scenery project is not a matter of coloring between the lines. Every project requires an asset library of plants, trees, grasses, volumetrics, buildings, bridges, roads, boats, trains, ships, jetways, vehicles, people, trash cans, dumpsters, transparent windows, furniture, GSE, fences, ground textures, building textures, seasonal textures, LM textures, Point of Interest objects and thousands of other bits that create the composite project. Many of those assets require special animations or other parameters to control time of day, seasonal and weather related display parameters. These assets need to be built and hand painted one by one, then stored in a comprehensive archive. This is what I have been doing for the last decade. All of those assets have been wiped out along with the composited builds. The aircraft in progress represent another massive investment in time, which has now been reduced to a pile of meshes and useless code.

In addition to the assets, all of the clever bits that make a sim more efficient and unique to its user have been destroyed.

All of the accumulated skins and freeware from a decade of collecting have been lumped in with the 1.2 million other files in what amounts to a digital landfill.

This is a catastrophic loss and un-recoverable. Even with the honorable and generous offers of help, I simply cannot allow my friends to fix what amounts to a deep personal loss. I have a love and passion for this work and am doing my best to come to grips with this invasion of my Intellectual Property. In my opinion this is a criminal offense, for which there should be remedy...but there is none. My feelings about this? Unless one has worked twenty hour days, seven days a week for a decade to have it erased without any recourse, it would be difficult to experience what I am feeling. There are no words to express my anger around this event. Every hour that I am spending gathering up the pieces add to the frustration, and this will go on for many months to come. To put it mildly, I'm in leg breaking mode.

I have taken on one project that was only barely started when I lost the data. Re-launching that project will require a minimum of backtracking, so this is the one that will occupy significantly less of my time than I previously allocated to the development work. I will not be doing updates or progress reports, I'll simply complete the project and release it. After that project is finished, I shall retire completely from flight simulation and move back into my real world projects, which will make my wife, who tolerated and supported this errant venture, extremely happy.

The work I have published on this site shall remain available as my small contribution to this community who have treated me with respect and kindness over these years.

Best to all, and many thanks for the years of support.

Gordon

[falcon409]

Many of us over the years have lost data, lost projects that were aaaaalmost completed and threw in the towel. . .for about a week or so and then went back at it. What most of us have encountered as "piddlers and tweakers" doesn't come close to what you've had to come to grips with.

I am sorry this happened to you and that's little solace I know, yet beyond what financial assistance I would muster up for a good friend. . .that's all I can offer. I hope that at some point all this becomes a very bad memory, eased by knowing that it's behind you and what you decide to do from this point brings you volumes of happiness. You certainly deserve it Gordon.

If you do jump in with both feet and pursue the woodworking you talked about. . .take screen shots and let us see how the other half lives.

[gman5250]

Many of us over the years have lost data, lost projects that were aaaaalmost completed and threw in the towel. . .for about a week or so and then went back at it. What most of us have encountered as "piddlers and tweakers" doesn't come close to what you've had to come to grips with.

I am sorry this happened to you and that's little solace I know, yet beyond what financial assistance I would muster up for a good friend. . .that's all I can offer. I hope that at some point all this becomes a very bad memory, eased by knowing that it's behind you and what you decide to do from this point brings you volumes of happiness. You certainly deserve it Gordon.

If you do jump in with both feet and pursue the woodworking you talked about. . .take screen shots and let us see how the other half lives.

I'm honored to call you my friend Ed, we'll have that beer.

[DC1973]

The work I have published on this site shall remain available as my small contribution to this community who have treated me with respect and kindness over these years.

Best to all, and many thanks for the years of support.

I'm fairly new here, GMan, but I just wanted to say that I'm so sorry for all that you've lost and how awful the whole thing has become for you. I'm only starting out in this field really, but if you should every choose to return, whatever assets I have built are yours too. All I can do really,

Best wishes for the future

[Sundog]

Best of luck to you GMan with all of your real world endeavors. May they proove much more rewarding and much less stressful.

Ken

[Mach3DS]

Gordon, I am totally caught up now on what happened. I did not understand that it was a COMPLETE loss. I am so sorry for this. If you should decide to change your mind regarding your complete retirement from flight simulation, I would be all for it, and would love to collaborate on a future clean sheet project with you. Perhaps not in the P3d/FSX universe. I want to publicly thank you and let those here understand what this means for this community. I reached out to Gordon a few years ago. About 5 years ago actually. I had started experimenting with repaints. But I had not the Photoshop skills to go along with the desire. Gordon shared his secrets with me. He literally took me under his wing. He literally laid it out on a clean sheet for me, step by step how to create art in Flight sim. He didn't tell me how to be creative, but he explained how it all worked. I then studied the SDK and it all started to make sense. I then came back to Gordon several times moving on to bump map creation etc. Each time he helped me. For Gordon to retire completely is a big personal loss to me, and will be to this community. I want to thank Gordon for his kindness and patience over the years. This student will be forever in debt. Thanks Gordon.

PS - If you would like to simply fly, maybe P3D/FSX is too painful to be in, I'd be happy to dart across the skies in DCS or some other sim with you! Stay away from creation and simply "hobby" for an hour or so every now and again.

Best wishes,

Rick