Storing More Than Files
So, once I learned that I could store very large amounts of data per row, I began concocting a plan to use web services to deliver data URI's of the images I needed to offline-stored web apps that would then store the base64-encoded data to the database for future offline use. For several days I was plagued with how to execute cross-site requests to pull back the images since the web app would be used offline. This was resolved by using JSON-P calls from the jQuery library to SOAP-based web services. If you haven't taken a crack at using this yet, you simply must. It was truly easy to do and several times had me shaking my head thinking, "This can't be this easy," but it was. It finally seemed that the strengths of each component were going to work in concert to perform beautifully. I should have known better, right?
You Should Know Better
After reading so much on the web from people that seemed to be asking my questions and getting, presumably, good answers, I didn't think I had left anything to guess work. What I hadn't anticipated, however, was that there would be so many people out there ready to provide answers that didn't really have the knowledge in practice to be qualified to provide those answers. So, when I read overwhelming feedback that indicated that you could store more than 50 MB of data in Safari and, later, that the secret to beating that limit was to break it up into multiple databases, I was feeling pretty sure of myself.
The first big thing to break was the single database at 50 MB. When I hit the limit, the insert just fails silently. Despite having error callbacks, it just doesn't insert and silently fails the calling routine. This lead to "neverending syncs" - a problem I faced using the localStorage caching mechanism in my last go-round. While I likely could have used a timer such as setInterval() to look for progress to cease and signal a timeout, it wasn't worth it because no amount of error handling was going to give me more than 50 MB of storage in that database on iOS.
Ok, so the next big break happened when I followed statements in multiple boards suggesting that the 50 MB limit was a per-database limit and that a somewhat-elaborate scheme of managing multiple database files could allow you to get around this roadblock. Armed with this, I recreated the data management layer to account for a master database that cataloged the various child databases being created and provided for simplified storage tables in each. It didn't take long, however for this data architecture change to prove just the opposite. At present, no amount of stringing together databases will work because there is a per-site limit in Mobile Safari of 50 MB, not a per-database limit. At this point, I was even more frustrated as I felt that I had made no more progress than back when I was using localStorage caching to handle the dirty work. And I was right.
After adding some debugging messages and console write-outs, I determined that about 5 MB of data was being read and written to the database when it starts to fail. With a heaping scoop of skepticism that this was more than a coincidence, I went head-first into Google and looked for answers. Sure enough, there was plenty of material to support what I was seeing: PhoneGap native applications have a 5 MB limit on the SQLite databases you create. After several iterations of, "How could this be?" I felt the wind being completely let out of my sails. I had worked a long day and it was now close to 11 PM and I had not taken a break for dinner yet. So, I hit the nearest Steak n Shake hoping that a Frisco Melt might at least eliminate one of my immediate woes.
Davide Saves The Day
Back to work and more Google searching in hopes of saving what sanity was left. I poured through a lot of links (some of which were blocked as "hacking sites" by our enterprise web filtering system) and was losing hope in 15 minute increments. That is when I came across a simple GitHub entry (PhoneGap-SQLitePlugin) for a PhoneGap plugin specifically targeting SQLite quota limits and my spirits were immediately raised. Italian IT engineer Davide Bertola created this plugin for the exact same reason I was searching for it. While the plugin isn't considered "complete" by his own admission, it was certainly feature-complete enough to provide the needed boost to my project.
The plugin took a little help to get compiled into the project as many of the "# include" statements follow a different syntax in Xcode 4.x than in the 3.x version and, at that time, there was very little instruction on how to implement the plugin successfully. In my hour of need, however, I contacted Davide via GitHub and was happy to have received a response about an hour later. His assistance mixed with some research into postings regarding PhoneGap plugins in the new Xcode led me to straighten things out the next morning.
You may wonder how Android did throughout these trials with iOS and it is worth noting that Android does not presently seem to be bothered by the limit though, at the time of this writing, I have not probed for an actual limit and am assuming the theoretical limit of "space available" to be an absolute maximum only. I also find myself mentioning Mobile Safari a lot when I am talking about these limits. It may escape a reader how specific this is. The full version of Safari on a Mac does not possess the same limits as Mobile Safari. While I recognize this may cause some readers to exclaim, "Duh!" I also think this is important to underscore. So, if you are writing specifically to target Chrome or Safari on a full desktop or laptop then you are good to go. But, honestly, who can afford to do that?
- As of iOS 4.3, Mobile Safari has a 50 MB SQLite limit that is per-site, not per-database
- PhoneGap (default) will support only 5 MB of SQLite storage
- PhoneGap plus the SQLite Plugin compiled into a native iOS app is a quick-win for some cases to allow mega-storage while reusing your web app
- You can have a native app on iOS and use the same base web app code for other platforms where quotas are seemingly unenforced or not present
- While a lot of developer help on the web is great (thanks again to Davide Bertola), you (read: I) can be foolishly trusting people that have not put theory into practice