Thursday, March 29, 2012

Configuration question

We do several imports into our database per day which may contain > 150k
records in 1 table, 50k in another and 10k in a third. These imports need to
run within a transaction because if any part of the import fails we need to
rollback all the data.
I recently got a Suspect database with a corrupt log file due to the very
large transaction file.
Is there any specific maintenance I should do or configuration setting to
better handle this? I guess I would like to clear the log and shrink it
after each day. Also, there seems to be an implicit table lock when the
imports are done. Is there a way to avoid this?
Thanks,
JoeTo keep the trans log down, you want to keep the transaction small. But in
your case, this is unavoidable. All you could do is to schedule this import
at low time. Also, you might consider dumping the data into staging tables
before inserting into the base tables.
-oj
"Joe" <J_no_spam@._no_spam_Fishinbrain.com> wrote in message
news:e3AY2XjQFHA.2252@.TK2MSFTNGP15.phx.gbl...
> We do several imports into our database per day which may contain > 150k
> records in 1 table, 50k in another and 10k in a third. These imports need
> to
> run within a transaction because if any part of the import fails we need
> to
> rollback all the data.
> I recently got a Suspect database with a corrupt log file due to the very
> large transaction file.
> Is there any specific maintenance I should do or configuration setting to
> better handle this? I guess I would like to clear the log and shrink it
> after each day. Also, there seems to be an implicit table lock when the
> imports are done. Is there a way to avoid this?
> Thanks,
> Joe
>

No comments:

Post a Comment