Skip to content

After restart logstash sincedb is not cleaned (sincedb that generated in previous running) #250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
godhelpsus opened this issue Oct 11, 2019 · 2 comments · Fixed by #276
Closed

After restart logstash sincedb is not cleaned (sincedb that generated in previous running) #250

godhelpsus opened this issue Oct 11, 2019 · 2 comments · Fixed by #276
Assignees
@kares
Labels
bug

Comments

@godhelpsus
Copy link

godhelpsus commented Oct 11, 2019 •
edited
Loading

I have an experience with the inode recycling problem, so I changed the sincedb_clean_after value like below.
sincedb_clean_after => "6 hour"

It is working fine.
But 6 hours after restarting the logstash , the old record which is generated previous running is not cleaned.
The record which generated current running is cleaned normally.

I think the logstash should clean old records in the sincedb no matter when they are created.

  • version : logstash-7.3.0
@andsel
Copy link
Contributor

andsel commented Jan 21, 2020

@godhelpsus I've some questions:

  • which is your mode? read or tail
  • if in read what value has file_completed_action
  • after the 6 hours, before restarting Logstash, did you still see the expired file?
  • after the 6 hours, if you stop Logstah witch is the content fo you sincedb file? If now you restart Logstash with --debug or --log.level trace command line switches do you see that input-file plugin discover again your expired file?

@jsvd
Copy link
Member

jsvd commented Sep 1, 2020

looking at the code that performs sincedb cleanup, it only happens when the sincedb is serialized to disk or when a file is opened.
In the scenario where you're working with a single event or other files being watched that aren't being written to, then the cleanup code never executes, only during specific eventst, like a shutdown.

@jsvd jsvd added the bug label Sep 1, 2020
@kares kares self-assigned this Sep 10, 2020
@kares kares mentioned this issue Sep 10, 2020
Merged
kares added a commit that referenced this issue Sep 25, 2020
@kares
Fix: sincedb_clean_after not being respected (#276)
20db56d 
In certain cases, in read mode sincedb is not updated.

There was no periodic check for sincedb updates that would cause the sincedb_clean_after entries to cleanup.
The cleanup relied on new files being discovered or new content being added to existing files -> causing sincedb updates.

The fix here is to periodically flush sincedb (from the watch loop).
Besides, to make the process more deterministic, there's a minor change to make sure the same "updated" timestamp is used to mark the last changed time.

resolves #250
expected to also resolve #260
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kares kares

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix: sincedb_clean_after not being respected kares/logstash-input-file
4 participants
@jsvd @kares @andsel @godhelpsus