[libFirebaseCppApp-10_1_1.so] firebase::firestore::FieldValueInternal::FieldValueInternal(long) #586
Comments
|
The code to send data to Firestore is very simple: One more thing I'll add, we have two sets of data we are storing, the UserData and the LevelData, and I am calling them both at the same time. I am not waiting for the UserData to finish before calling SetAsync on the LevelData. Could this be the issue? |
|
Hi, Thank you for filling up the ticket! The team is not available for support during the holiday break and we will get back to you in the new year. Happy holiday! |
|
Some additional information: Crashlytics reports -
|
|
Hi @nitzanwilnai , Could you provide the crash report in detail? For example what is the error message when crashes? |
|
I am not sure what you are asking. I included all the information above. The messages in Google Play and Firebase and the stack traces. There is no error message, only the crash report. We know that on the user device in production the game simply exits. Note that every single user successfully saved their data to Firestore multiple times. Most had 20+ minute sessions where they saved the data dozens of times without issue before the crash happened. If it makes it easier I can combine all the info here: Firebase: Google Play: pid: 0, tid: 17430 >>> com.fourxp.mergeup <<< backtrace:
backtrace:
backtrace:
backtrace:
pid: 0, tid: 14281 >>> com.fourxp.mergeup <<< backtrace:
pid: 0, tid: 3325 >>> com.fourxp.mergeup <<< backtrace: |
google-oss-bot
added
needs-attention
|
We also believe that the non-firebase issues on Google Play are also related: For example the various But none of them have any stack trace. Note also that we have not been able to reproduce any of these issues locally, they only happen in production. |
|
Note that updating to Firebase 10.3.0 did not fix the issue. |
|
This is likely a duplicate of #569 (the android global refs exhaustion issue). If you have a lot of Firestore documents in memory with a lot of key/value pairs and/or large arrays in them then this can exhaust the pool of 51200 global references, resulting in a crash. We are actively working on a fix for this (firebase/firebase-cpp-sdk#1176) and hope to have it released in the next Unity SDK, planned for late January or early February 2023. |
DellaBitta
added
needs-info
|
So our documents are fairly optimized (bytes instead of ints, single letter variable names, etc...) |
google-oss-bot
added
needs-attention
|
I've looked at your crash reports more closely and can confirm 100% that you are experiencing the Android "global refs exhaustion issue" (#569). The fact that the crashes are occurring in Now, it's not the size of the documents, in bytes, that matters, but rather the number of key value pairs and the number of elements in array and map values. Each value and each array/map element consumes a JNI global reference, along with each Firestore object (e.g. DocumentSnapshot, Transaction). Unfortunately, there really is no workaround other than reducing the number of Firestore objects in memory at a given time. If you have documents with large array or map values, then avoid keeping those documents in memory. The upcoming fix will likely address this crash though. It will, unfortunately, be a month until the release of the fix though. I'm going to close this issue since it is a duplicate of #569. Please follow that issue for updates. We can still discuss here in this issue if you have follow-up quetsions. |
|
Can you explain to me a bit more what you mean by keeping these documents in memory? Or are you talking about the copy of my [FiretoreData] class? From our logs, it seems like there is a memory leak somewhere in Firebase every time we send our FirestoreData class to firebase, and that after X amount of times we exceed the table you are talking about. Thanks! |
|
Oh, all I mean is holding on to DocumentSnapshot objects. Make sure you null out references to Firestore objects once you don't need them anymore (or let them go out of scope so they will be garbage collected). If your app is well-written, there may not be any opportunity here to do that though. |
|
So in my SendUserDataToFireStore code here: ` Do I need to save a reference to the docRef, then set it to null once the ContinueWith returns? |
|
The DocumentReference only consumes 1 global reference. So unless you have a LOT of DocumentReference objects alive at once then it doesn't matter. But to answer your question, you should set docRef to null ASAP, unless it just goes out of scope, which has the same effect as setting it to null. So probably the cruft of setting it to null is not worth it. The more interesting object is the |
|
So yes, we have two big objects, userData and businessData, and they both have a few arrays, some of them quite large. In one we had an array with 1024 integers. We also have nested arrays of other FirestoreData classes. We have tried to reduce everything to as small as possible and so far are seeing fewer crashes. That said, I still don't understand why the crashes happen after 20+ minutes of gameplay and dozens of SetAsync calls if the memory gets freed once the SetAsync call is done. |
|
Here is our userData class: namespace MergeGame } You can tell that at some point we actually read the Firestore doc and started changing our names to the shortest possible. ;) |
|
Our BusinessData: ` namespace MergeGame } |
|
Also here is a log from a user that crashed. We only send the data once the user is inactive for 2.5sec. So as long as the user is interacting with the application we are not sending any data. 1 |
|
Yep, those objects look like heavy global ref consumers. Probably what is happening is that the Firestore C# objects are accumulating in memory, even though they are no longer referenced. Eventually, the garbage collector runs, destroys the C# objects, and those objects' finalizers are run to free the Android global refs. But if the number of global refs hits the 51200 hard maximum before the garbage collector runs then the exhaustion occurs, resulting in the crash. That's probably why it takes 20 minutes or so before the global refs accumulate to a problematic level. The only workaround that I can think of would be to call |
|
Is Firebase using its own garbage collector or is it using Unity's? Can I manually cache and re-use the document refs instead of creating new ones? |
|
Firebase is not using its own garbage collector; it's using Unity's. No, there is no way to manually re-use the document refs instead of creating new ones? |
|
In that case, would calling System.GC.Collect() or Resources.UnloadUnusedAssets manually at opportune times solve the issue? |
|
It's worth a try, although I don't know for sure. It depends how the garbage collector responds to those calls. |
|
FYI, unfortunately calling: |
|
The only solution we found is:
At least on some devices, there doesn't seem to be any clearing of the reference pool, ever, during gameplay. Possibly on all devices. To me it looks more and more like you guys have a memory leak in the code. |
|
Interesting. It's entirely possible that there is a memory leak in Firestore. Would you be able to use something like Android's "Memory Profiler" to see if you can get some more details about the memory usage of your app, and/or find some evidence of the unbounded memory growth? This information could point to the part of the Firestore SDK that is leaking memory. https://developer.android.com/studio/profile/memory-profiler |
|
Hi, I do not see any sort of leak in the memory profiler. |
|
Okay. I was hoping the memory profiler would give some useful information. Thanks for trying it out. You may be right that there is a leak somewhere in Firestore, but for now we will focus our attention on fixing the exhaustion of the Android global reference pool. The fix is still expected in the next release. |
|
Hi, another question. You mentioned an array of X elements would be X+1 references (if I understood you correctly.) |
|
A String is just 1 global reference (the reference to the java.lang.String Java object). |
Hi,
About 15% of our users seem to have Firebase related crashes, it seems when saving to FireStore.
After uploading symbols to Google Play, we managed to get the following crash data in the Google Play Console:
[libFirebaseCppApp-10_1_1.so] firebase::firestore::FieldValueInternal::FieldValueInternal(long)[libFirebaseCppApp-10_1_1.so] firebase::firestore::jni::Global<firebase::firestore::jni::Object>::Global(firebase::firestore::jni::Object const&)[libFirebaseCppApp-10_1_1.so] firebase::firestore::FieldValue::FieldValue(firebase::firestore::FieldValue const&)Some additional information:
After adding symbols to Firebase, we have the following crashes on crashlytics:
All three crashes are happening when saving our UserData to FireStore, and they are each happening to different users. So a user who experiences one does not experience the others.
Any ideas?
Thanks
The text was updated successfully, but these errors were encountered: