Add a comment

 

Re: The joy of sets

During a discussion with the lead developer for a client's batch systems I discovered that he was thinking more along the lines you've just described, only in-database rather than in-memory. He would question why you had a primary key on the products table at all - why not simply remove the duplicates once the batch had finished? You would naturally suspend indexing and constraint checking during batch operations so why is your PK exempt?

The compromise was to do parallel batch loads into one set of unconstrained tables and apply the change set to your "relational" tables once the contention is under control.

His argument went a step further, though: since there was no other mechanism for creating products or relationships to those products why ever introduce constraints in the first place? By way of an example he demonstrated how one transaction (pretty much exactly of the type you describe) resulted in the same constraint being checked 6 (!) times for the sake of referential integrity that was already implicit from the source data.


Re: The joy of sets


Title
Body
HTML : b, strong, i, em, blockquote, br, p, pre, a href="", ul, ol, li, sub, sup
Name
E-mail address
Website
Remember me Yes  No 

E-mail addresses are not publicly displayed, so please only leave your e-mail address if you would like to be notified when new comments are added to this blog entry (you can opt-out later).