Jump to content

Easier method for deleting oldest lines from a data file as it grows?


Recommended Posts

I'm using Write To Data File script step to keep track of the outcomes of batch transactions in a text file. It's basically a log that users can view to see line-by-line what happened when they ran a batch process (which records succeeded, failed, queued for retry, etc.).

Entries into this log have a lot of value to users in the short term, but have little to no value to them as they age.

I'd like to purge the oldest entries (at the beginning/top of the file) when the file reaches a certain size. For example, if I want the max file size to be 128K and the new entries make it 130K I'd like to remove the oldest 2K of text in the file at a ¶ or line feed (so I'm only deleting whole lines, not partials).

The only way I've figured out how to do this is to read the entire file into a variable, use the Right() function to grab the rightmost 128K, apply some logic to locate the return, then write over the file with the new data in the variable. Is there an easier way of doing this? I feel like I'm taking a Rube Goldberg approach.

Thanks in advance!

-Kent

Link to post
Share on other sites

Putting all other kinds of potential issues aside for a moment, my main worry with the method you describe would be leaving a tail of old data when the new data is shorter than the current contents of the data file.

What would be the advantage of managing data stored in an external text file, when you have a database application at your disposal? If, for some reason, users prefer to view the log data in a text file, the simplest method would be to export it - overwriting the previous file.

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Who Viewed the Topic

    1 member has viewed this topic:
    TerenceE 
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.