Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upHow to reduce the size of the jar file by excluding language profiles? #27
Comments
This comment has been minimized.
This comment has been minimized.
Hi @seinecle, thanks for your question. Are you talking about limitation of RAM (1) or hard disk space (2)? For (1): It is enough to simply specify only those languages within the For (2): Actually, I have not yet thought about this use case of limited hard disk space. One way to overcome this could be to write a custom Gradle task that allows to specify the languages that the jar file should contain while building the jar file in a subsequent step. This would require to not try to load the language models of the excluded languages into RAM, otherwise an exception would be thrown that the requested language models could not be found. By the way, why are you using the old Maven-based version? Because the jar file is smaller? |
This comment has been minimized.
This comment has been minimized.
Thanks @pemistahl ! My issue is the second case (hard disk space). I am integrating lingua in my Java project by using the maven dependency:
Not sure my use case is a common one. For reference, the optimaize library (not maintained anymore - thanks for mentioning lingua in an issue, btw!) is less than 6Mb large. I had to revert to it because lingua can't fit my allocated memory space. |
This comment has been minimized.
This comment has been minimized.
What kind of environment are you working in? Android? Is there any chance of increasing your hard disk space? As I said, I would have to write a new Gradle task for customizing the build of the jar file. But I do not have much free time for coding at the moment. So your best option currently is, as I said, to unpack the jar file yourself and remove the language models you do not need. |
This comment has been minimized.
This comment has been minimized.
The environment is an AWS Lambda, which has a hard limit at 250Mb on the unpacked code. If I remove some folders, can it cause NPE when using the lib? |
This comment has been minimized.
This comment has been minimized.
If you specify languages in the |
This comment has been minimized.
This comment has been minimized.
ok thx! |
seinecle commentedFeb 24, 2020
•
edited
I need to run this lib in a memory constrained environment: less than 200Mb for the unzipped package. How can I exclude rare language profiles from the library?
An alternative: can the memory size be significantly decreased by minifying the json files used for each language?
Note: I am using the maven build of the lingua.