Replacing NDK

[vitalnodo]

Hello. I'm passionate about your project. As a next idea, you mentioned that it would be good to investigate the feasibility of replacing NDK with zig cc toolchain. How far did you go in your investigation? I saw this.

[akavel]

Honestly, I don't even remember I'm afraid 😓 I actually don't even recall sending this email that you linked 🤣

FWIW, dali/hellomello is in a somewhat sorry state from maintenance perspective at the moment. Part of the problem is, at the time when I was writing, Nimble didn't have any kind of "lockfile" for its package management, so I couldn't easily shrink-wrap a buildable set of packages. As a result, at the moment I myself am not even 100% sure how and which versions I would need use to build hellomello :( BUT, I also recently noticed somewhere, that apparently Nimble did get some kinda lockfile format. So a thought of coming back to hellomello+dali to try and at least link together some versions of things that would work does in fact float in the back of my head now. Problem is, I have waaaay too many such exciting ideas floating in the back of my head - and honestly, I'm not super sure I'll get to this one over the next 5+ years :/

Now, given the above, seeing you say you're excited about the project really makes my heart go giddy with excitement too 😅 I wonder if doing something like trying to prep up a lockfile (https://nim-lang.github.io/nimble/workflow.html#nimble-lock) for dali and/or hellomello would be something you'd be interested to explore? I'd LOVE to support you in this if you'd like to try. Personally, I think it would be a great and necessary first step towards trying zig cc. Honestly, I think it would be a great and necessary first step to do actually anything with this project at all..... 😅

There's also one extra thing I must note out of honesty: personally, I'm no longer playing with Nim currently. I loved it (and still do) for many things, including how the code looks, how it approaches macros, and also a bunch of others. I struggled somewhat with stability and quality of it, but I was able to live with that. Unfortunately, at some point I hit a roadblock with its concurrency story - I found this area cripplingly (to me) underdocumented. This ended up finally pushing me to Rust, which is currently my main hobby-time tech stack. That said, I do actually miss the syntax and simplicity of Nim when coding Rust. In context of our discussion, this also means I've actually did some first inroads towards trying to port hellomello+dali to Rust... I just now realized I didn't push that branch out, so I just did it: https://github.com/akavel/dali/tree/rust1 It's super early FWIW, but IIRC it does pass the core initial tests of assembling simple .dex files. That said, if the Nim version can be made to run, it was at a much more advanced stage of development than the Rust one is ATM. And, my development of the Rust branch is also very spotty, currently on a hiatus (though I'm recently pondering trying to push it forward a bit, so who knows what will happen).

So, that's what I can tell at the moment. Sorry for a rather chaotic reply. What are your thoughts on that in this context? Also, I assume you saw the NimConf video I made in the past, right?

EDIT: I also want to clarify I'd be fully and super-excitedly supportive of any efforts to push dali/hellomello towards Zig ecosystem (assuming, obviously, the license is kept). I honestly think it'd be a perfect and natural direction for this project, and it really was on my mind a lot. Yet, at the same time, I'm a complete zero in Zig, so can't help much from the Zig side of things. And at the moment, my personal decision of my personal time investment is just towards Rust 🤷‍♂️ But, I'm absolutely more than happy to support it from the Android/Dex side of things (as much as I myself don't know a lot about that), and primarily from the side of things of trying to understand/do things in this project's codebase.

EDIT2: do you maybe have a mastodon account? For faster discussion, I'm happy to make contact there, I'm at: https://merveilles.town/@akavel

EDIT3: Also, I distinctly remember I was seeing an open road ahead to port apksigner to Nim too. I found somewhere some blog where some ppl were showing the format of pkcs7 file, and it seemed really easy as long as only limiting it to the parts we need for making a signature file. But I'm not super sure where I might have the link at the moment unfortunately 😔

EDIT4: In asking about whether you seen the video, I want to also make you aware of the https://github.com/akavel/hellomello/tree/flappy / https://github.com/akavel/hellomello/tree/flappy.01 branches 😜 (again, I don't recall which one of them was actually working...)

EDIT5: link to the video just in case: https://www.youtube.com/watch?v=wr9X5NCwPlI&list=PLxLdEZg8DRwTIEzUpfaIcBqhsj09mLWHx&index=12

[vitalnodo]

Thanks for the warm and friendly reply.

Yes, making the project build out-of-the-box again sounds like a great goal. For the basic hello world example, I had to regenerate the keys and make small changes like Bundle → Bundle: Bundle, which could already scare people away from trying things. By the way, based on some testing, it turns out you can compile it just fine with only the headers from the NDK:

zig cc -target arm-linux-musleabi -Oz -fPIC -shared \
  -I /opt/android-ndk-r19c/sysroot/usr/include/ \
  -I /opt/android-ndk-r19c/sysroot/usr/include/arm-linux-androideabi/ \
  -o libhello-mello.so exp/hello-jni.c
mv libhello-mello.so lib/armeabi-v7a/libhello-mello.so

So basically, just the headers are enough — Zig does a great job compiling the rest.
Maybe that means adding bionic support in Zig wouldn’t be that hard after all — or maybe I’m being too optimistic and just got lucky. Probably it would be useful to analyze https://github.com/ziglang/zig/wiki/Updating-libc Probably at the current step there is no problem with unique linking details or something.

As for the flappy bird example — that’s trickier. It depends on third-party packages like yglukhov/android, and I couldn’t find where android/graphics/paint comes from. So I wasn’t able to build it yet. I assume it’ll take more investigation and possibly installing extra stuff. So yeah, a lockfile would be helpful here — I’ve started looking into it, though maybe you’ll recall it faster than I can investigate it.

For now I’ll probably keep posting here publicly (maybe someone will be also interested), but maybe at some point I’ll figure out Mastodon and join there too. And maybe I should try creating a Zig-based implementation? Like an AGPL binary that could be used to build other projects under any license?

Pretty sure I’ll have more questions as I go. Thanks for your updates too.

[akavel]

As to android/graphics/paint, I thought I maybe added it to my fork of yglukhov/android, but I don't see it there unfortunately: yglukhov/android@master...akavel:android:dali 🤔 there's non-zero chance I might have had something only locally, and forgot (or was too tired) to push it out 😬😓 from an older machine of mine...

[akavel]

For sure I also have a fork of jnim itself; possibly this branch: yglukhov/jnim@master...akavel:jnim:dali or another one...

[vitalnodo]

After some days of despair I managed to get it running
https://github.com/vitalnodo/hellomello/tree/flappy-update
https://github.com/vitalnodo/android/tree/master-fix

[vitalnodo]

[akavel]

😍

[vitalnodo]

I just stumbled on this: https://github.com/desttinghim/cyborg/
it’s basically what I was planning to do

[akavel]

I just stumbled on this: https://github.com/desttinghim/cyborg/ it’s basically what I was planning to do

Oh cool! 🤩
From a quick glance, not knowing much Zig, it seems to be an assembler from the *.dexasm format to signed *.apk files, yes? Would you be planning to make it also allow linking with *.so files cross-compiled for arm from Zig? or does it already allow that and I didn't see?

May I kindly ask if you'd be OK to open PR(s) with your current state of improvements/fixes to dali etc., assuming you might be switching your gears to work on a "cyborg" repo fork from now?

[vitalnodo]

Right now I'm still exploring what's already implemented in cyborg — that's why I opened an issue there to get some context from the author. So far, it looks mostly focused on .dex handling, yes.

At this stage, I feel the need to experiment more with .so files — something a bit more complex than a typical hello world. That might even lead to opening an issue in Zig itself about Bionic support, though I hope that musl, some compiler_rt bits, and other tweaks might be enough for simple cases. For JNI wrappers, I’m also considering using Zig’s metaprogramming features to auto-generate them. In the long run, maybe this could evolve into something like bun.sh, but for Android. But before all that, I still need to look through what's already there, see what can be improved, update the code to the latest Zig version, and generally get a better feel for the current state.

I submitted a PR to android for flappy bird, and a small fix to dali to make the hello world example build.

The other changes I’ve made are mostly related to regenerating certificates and adjusting package dependencies — so I’m not entirely sure how (or whether) to submit those as a pull request.

Also, I think that since the flappy branch is actually more advanced and interactive, it might be worth updating the README to include the instructions from the video — and maybe even consider making that branch the default one.

[vitalnodo]

Just dropping by now that there’s a more relevant issue:
ziglang/zig#23906

It sounds more optimistic now — there’s mention of some libc-abi-tools, the abilist format looks promising, and Zig recently gained cross-compilation support for NetBSD and FreeBSD too.

Meanwhile, I started craftdex — a .smali → .dex assembler written in Zig.
It still needs a lot of work, but while building it, I started wondering:
what if Zig AIR could be used to generate .dex directly, similar to SPIR-V codegen — like in this example?

So instead of using macros, .smali, or external tools, it could be embedded into the compiler, allowing a limited subset of Zig functions to be compiled into .dex using a specific calling convention.

Any thoughts?

[akavel]

I definitely thought about maybe adding a .dex backend to the Nim compiler; just I assumed this to be a notably harder / much more time consuming task, that I simply didn't see myself having enough free time for... also I preferred to go with the "smallest thing that could possibly work" approach; especially while not even knowing if it can actually work 😂

As for Dalvik assembly, I personally wanted to use a higher level programming language, so I wanted to do as little assembly as possible - just a smallest possible shim.

[akavel]

@vitalnodo I want to thank you for creating this issue, and for the work you did on "recovering" the "flappy nim" project, and for sharing all of that with me, and for sharing your energy. It helped me get back some hope and joy for this project, contributing to me recently starting again slowly chipping on the Rust version! thanks!! (That doesn't mean I'm in any way less than before endorsing your attempts at exploring any other approaches, through Zig or otherwise; personally I'd be more than happy if there was more than one lightweight indie way to build an Android apk without touching Java!)

[vitalnodo]

FYI, here's another attempt, based on some forum post. This one uses fasmg. There's some boilerplate and it's not really finished, but might be interesting and it contains some examples
https://github.com/vitalnodo/forgedex
Potentially it could lean more on fasmg's CALM instructions to cut down the boilerplate. I noticed you were trying to build something like a macroassembler in the rust branch and it seems like a related approach, if I understood correctly. I'll probably also try to contact the author of that forum post and https://github.com/MaoKo/gDalvik might be interesting