Btfj.dat – huge size and memory consumption
- josef.sabl
- Member | 153
Hi, how is btfj.dat supposed to behave? It is bloated (500 MB) after few weeks of running in production. It also appears to cause troubles with out of memory fatal errors as scripts consume more and more and when fatal error occurs, it is usually in FileJournal.php. What can cause it to be like that. What are we doing wrong? Thanks for any advice.
- Milo
- Nette Core | 1283
The btjf.dat file is a B+ Tree index for storing tags and
priority cache parameters. It makes cache very fast. Its size depends on how
many items with tags (or priority) you store into cache. The file is created by
FileJournal.
The 500MB is a pretty big index, but it should not behave oddly as you describe. There is a long-persist bug in the FileJournal (last report) and it is hard to find.
So, you don't do wrong anything, I guess.
- josef.sabl
- Member | 153
Thanks a lot for your response.
Let me ask one more question, please: Does filejournal has some fallback or self-cleaning mechanism that would throw away really old entries? I mean when I create an app that will push items into cache with tags like “entry” . $timestamp without any expiry, will the file grow indefinetly? Will it have some impact on the memory consumption or io operations? What if I set the expiry to those entries, will they be purged.
And on related note, I noticed that cleaning cache doesn't do anything to filejournal's file size. I have to clean cache, delete file and clean cache again, or else I get bugs :-)
Thanks again, appreciated!
And good luck hunting that bug :-)
Last edited by josef.sabl (2014-11-26 15:27)
- Jan Tvrdík
- Nette guru | 2595
The FileJournal uses 4kB for each node, therefore size 500 MB means that it contains about 128 000 nodes. Can you guess how many tags / items to you store in cache?
- Filip Procházka
- Moderator | 4668
If you have so much data, it would be probably wise to start using something that might be a bit faster, have a look at Kdyby/Redis journal.