# Coding

Work in progress
Wednesday, October 21, 2020

Protocol buffers, Google’s own data exchange format, love em or hate em. They don’t work particularly well on Android because the proto compiler spits out enormous blobs of Java code. Even empty messages will already result in around 15Kb of source and declare more than four dozen methods. Raccoon needs to implement 75 message types in order to talk to the Playstore. That comes down to a bit more than 3 Mb of uncompressed bytecode, scattered across 310 class files with more than 70k methods.

When they came up with Dalvik and the DEX format, they pulled a Bill gates: 65k methods should be enough for everyone. Surprise! Who’d ever think that there would be apps including giant blobs of generated parser code.

So, I don’t want to build a multi DEX APK. Luckily, there’s a protobuf-javalite runtime, promising a smaller footprint (it’s actually what the Playstore APK uses as well). Surprise! I still exceed the 65k limit, plus the normal and the lite runtime don’t seem to be API compatible, so I can’t make a raccoon library that works out of the box on PC and Android. Thanks Google! You really put an effort into making Android a compatibility hell on every level!

Friday, October 16, 2020

Gradle is annoying me. I got an artefact that depends on another artefact which I got in three different variants (there might be more in the future):

  • coon-proto-full (2.4 Mb)
  • coon-proto-slim (555 kb)
  • coon-proto-nano (550 kb; lite runtime)

These artifacts provide protocol buffer parsers required for communicating with Google Play. Exactly one of them is required. Slim and Nano implement fewer message types (Full exceeds some resource limits on Android), but otherwise all three have the same API. The idea is that they are drop in replacements for each other. Pick and choose, the rest of the library is agnostic.

With Maven, there are optional dependencies, Gradle has this very elaborate “feature variant” concept. Really, by now, I’d be happy to simply omit those artifacts from the dependency graph, but then Gradle can’t build the APK downloader library itself.

Friday, October 9, 2020

Well, that was a waste of time. Raccoon and Google Play communicate with each other by exchanging protocol buffer files (think: binary XML). The problem with protocol buffers is that Android only supports 64k methods per DEX file and the protoc compiler exceeds that limit effortlessly. Of course, Android also offers a dirty hack to put more than one DEX file in an APK, if you are willing to bundle yet another compatibility library. I’m not, so I thought that switching to java-proto-lite would be a good idea. Lo and behold, the proto parser code shrunk by a magnitude! Sadly, the lite runtime doesn’t work with the proto2json export library. So yeah, lean code or being able to cleanly export pb to json. Tough choice.

Monday, October 5, 2020

HOWTO: Running Java code directly on Android (without creating an APK)

A step by step instruction for compiling a Java program into Android executable and using ADB to run it.

Saturday, October 3, 2020

I really wonder when Android is going to implode under all of the technical debt, it’s engineers have been diligently piling up. You can’t even write a simple “Hello World” app anymore that just runs on all versions of Android. Well, you probably can, but you can’t publish it on the Playstore unless you target at least SDK level X (changes every year). In order to do so, you are forced to bundle a giant compatibility library (congratz, wasn’t the whole idea of the DEX binary format and that idiotic resource system to squeeze every byte out of the APK files?) and half of your code will deal with figuring out on which OS version it is running and how it should behave.

There’s really no part in Android that’s not an awkward compatibility hack. Playstore included.

Friday, October 2, 2020

The annoying thing about this whole HttpClient v5 migration is that I thought, I finally had a libraccoon release candidate ready (I have been working on modularizing the APK downloader component for the past months). Now I find all kinds of bits and pieces to be broken (thank heavens for unit tests!) because the new HttpClient library works differently in subtle ways.

Thursday, October 1, 2020

Looks like I finally finished migrating from Apache HttpClient v4.x to v5.0. Two solid days of work, but probably worth it in the long run. I still don’t get the new design of the HttpClient interface. If you are using it without a Handler, you get a HttpResponse that you can’t close without an ugly cast, potentially either resulting in a resource leak or a class cast exception. The examples all say to use an instance of CloseableHttpClient instead (from which everything is subclassed), but what’s the point of having an interface if you have to use it’s one implementation anyways?

Wednesday, September 30, 2020

So, after being utterly pissed about what Google did with HttpClient v4, I spent yesterday on upgrading to HttpClient v5 (did not notice it coming out of beta in February). Good news is, my SSL spoofing code still works. Curious news is that Google changed something about the logging process (again). Previously, if they detected that you are not on Android, you’d be stuck in an endless CAPTCHA loop. Now you just get a plain HTTP 403 with a details message of “Bad authentication” (which is kinda accurate). Looks like what I initially thought to be a red herring was actually just a bug. Of course, this keeps me from properly testing the new CAPTCHA code path…

Anyway, no reason to be alarmed, this doesn’t affect Raccoon.

Tuesday, September 29, 2020
Tuesday, July 7, 2020

Gnarg! Why is there no way on vanilla Android to view the packagename of an installed app? List My Apps to the rescue!