Professional feedback please: yea or nay on a migration strategy

[AHunter3]

I work for a company using a high-volume database, at the core of which are two main tables, Orders and Line Items. The system in its entirety is massively complex as well as old and creaky, so with their affirmative blessing I rebuilt from scratch. New system is being beta tested.

The following may or may not be my idea; it may or may not be an idea proposed instead by someone else at my company.: I have asked this elsewhere on message boards where I am more active. I hope that simulposting it here is not considered bad form, but I'm trying to get a decently wide set of responses.

What if, instead of siphoning off some workers' time & energy to put some trial records into the newly-rebuilt system, and then, once it seems to be working reliably, do a fresh import of all the data from the old sys and switch to full-time use of the new, we instead have the workers gradually start using the new system as the "real" system?

I mean, let's say beginning with orders of a certain type (let's say orders for electronic devices of a certain type, with warranties of a certain period, placed by new customers only, and by customers not ordering any other items at the same time), we would enter those orders in the new system ONLY, all other data entry taking place in the old system. We would do that for 4 days and if it goes smoothly we would then also start entering orders of a second type (let's say an order otherwise exactly like the first type except electronic devices of a different model series would constitute this second type; or perhaps the second type would be customers who are purchasing a second item at the same time). This second type would now be entered ONLY in the new system, the first type would CONTINUE to be entered ONLY in the new system, and all other remaining types would continue to be entered in the OLD system.

For any given order type, if there were problems within the new system, we would halt order entry in the new system for that order type and go back to doing it in the old system untll that problem was sorted out. Then we would try relying on the new system for that order type once again and after a few successful days go on to a third, fourth, etc, order type until eventually the new system is the only system being used for entering new orders, of any type.

=======

Your reactions, thoughts, and comments please.

[LaRetta]

Hi Allan,

My only concern would be updating your customer base while this testing was going on. Even if only new customers, what if one custormer ended up ordering two types of orders (or would that ever happen)? Might you be dealing with updating customer information in two databases (if their phone changed for instance)? Synching that customer data, as well as order and product information might get complex, particularly serials for orders and lineitems. How would you determine who took precedence if a record was changed in both systems? Product prices changed? Documents mailed? It sounds like a possible mess to me.

I wouldn't necessarily recommend against it, although I don't believe I would do it. But I would be very cautious (as you appear to be so that's good). I would be inclined to run parallel systems before I would split it up.

LaRetta

Edited January 23, 2008 by Guest
Changed 'included' to 'inclined'

[AHunter3]

Hi, LaRetta, thanks for responding! Yours is a name I wish I still saw as often as I used to.

The main advantage to the proponent(s) of this approach (sorry for the deliberately vague wording, we are trying to avoid coloring responses by possible reactions to AHunter3 per se) is to not do redundant data entry -- to enter a few types of orders in the NEW system which would NOT be entered, at all, at any time, in the OLD system. No order gets entered twice.

The concerns you have raised, about messiness and confusion, are prominent among those voiced by the opposition who are less enamoured of this approach.

Any elaboration on your answer that you might care to provide, in light of that, would be appreciated.

[AHunter3]

Time to end the suspense and explain what's going on. I appreciate all the feedback.

Here's the deal: My company's owner/executive is the person who has proposed this plan as an alternative to my standard plan of having the regular users put a small percent of the live data (5-8 %) into BOTH systems, enough dual data entry to give the new system a good solid shakedown cruise, after which the data in the new system would be dumped and all data from the old system reimported, and from then on the new system would be THE system, with the old one available only as a reference.

The company owner has been reluctant to let workers spend time beta testing (they are understaffed, it takes them away from "real" data entry) and is also reluctant to hire some former employees in the area who left the org on good terms, who know it well enough to do beta testing (this idea was floated).

I said: hell no, that is totally not the way it is done, you don't DO that, that would be like driving down the highway with one leg in the window of a Pontiac and the other leg in the window of a Chevrolet in the next lane over. Really bad idea. You hired me, but I am a professional. It would not be ethical of me to do this.

He is not accustomed to this. He said (not directly to me, but through the person to whom I directly report): I've worked with databases and people do this, it's a reasonable strategy. I said: Umm, no it isn't. I said: would it be useful if I poll my peers to get some second opinions?

Well, I got them. Thank you. Via the TechTalk Digest, FileMaker Cafe, Worldwide FileMaker Forums, the New York FileMaker Developers' Group, and for a more general pool of database geeks the Straight Dope Message Board, I got replies. A surprising 3 people out of 22 respondents said it might be OK, although 2 of those three went on to say it's not how they would do it. Three people said it was a recipe for disaster. The overwhelming response was "you should do parallel testing instead", and a great many specific concerns were voiced about the kinds of things that happen when your "live data" is not in one system or the other but awkwardly distributed between the two, concerns that mirror my own thinking.

Thank you again for giving me ammunition to claim that I'm not holding an unusually conservative opinion here, and that pretty much any other developer would tell him the same thing.

Now I have to go explain to this guy why being a hospital administrator does not authorize one to tell the surgeon how to do the surgery.

[FM Tid]

I was a professional systems analyst and mainframe programmer for years working for various high-profile companies. As such, working both WITHIN a Quality Management System (QMS), and for other companies that had never heard of such a thing, there is one thing they would be in unanimous agreement on:

NEVER EVER test a new system with 'live' customer data. The scope for absolute disaster is one small wrongly-coded 'switch' or 'flag' away. I speak from experience - I once believed I had tested a new system to destruction, and then went on holiday. During that week, something went wrong and all hell broke loose. And this was a system change that was 95% tested!

Do whatever it takes, but convince your company manager that this is an total NO GO. Don't even think about it! Only if he can afford to potentially lose all his customers and start rebuilding his company from scratch should he entertain this notion. Professional and commercial suicide is not too strong a word for it.

Edited March 4, 2008 by Guest