Thank you for your suggestions here. Our relative colleagues are working to improve the feature of 3-rd party Note Apps. Please be patient.
2 Likes
Really? I was unaware there was work being done to improve 3rd party note apps. I look forward to the firmware update in that case. Can you give any ETA on that, or is it just “in the future?”
This sounds to me like the average “we’re always looking at ways to improve our products” stuff. I wouldn’t expect any groundbreaking changes in the foreseeable future, simply because I believe it is technically very hard to get third-party note apps to work properly with the current software architecture, unless those apps use the official Onyx Boox SDK.
I am sorry to inform you that this will not be available in the November firmware update. We do pay high attention to this problem and already worked on this.
This is pretty close to what I had/have in mind, except just refresh screen after a couple of seconds of no stylus activity. It will require auto-disabling touch on stylus activity though.
Even just exposing the functionality for toggling raw stylus input mode in /sys/ is enough (a shell script to auto-toggle on stylus activity for certain apps is easy enough), but I couldn’t find it (theoretically, one can already accomplish this by making an appropriate service app, probably requiring root unless the SDK actually sends broadcast events every time stylus enters/exits screen).
Or just grab the apk and edit the smali like I did: https://youtu.be/3eHHH9-QHNk
Not sure if it’s visible or not, but I made it so that it pauses TouchHelper after 2 seconds of no activity to refresh the screen, so it’s not as wonky as you mentioned (if you’ve ever used stroke smoothing, you’re probably already used to it).
It actually doesn’t take that long to implement just the handwriting part. The hard part is figuring out where to insert things (easy if you attach a debugger).
The thing with doing it yourself and distributing hacks like this is that Onyx might no longer feel as compelled to make an effort (there are already existing solutions, why bother?), and other companies might just take it, rename it, and sell it off as their own (called BoyueNotes or something, assuming they use very similar SDKs).
3 Likes
Wow, this is super interesting! I know very little about Android development specifically, I dabbled in it a while back when I was learning Java, but not enough to have tons of experience. My first instinct was to decompile an apk and see if I could implement the API for it. I didn’t even think of using a smali hack at all.
Do you have a Github or git repo with any of this in it? I totally get the hesitation about distributing hacks like this, which IMO is why it would be such a good idea for Onyx to open-source most to all of the software running on their devices. If you’re uncomfortable with sending me that, could you provide any tips where to start, if I want to try a similar project myself?
No, but it’s pretty easy to come up with.
Just get an open source drawing app (I think I used Simple Draw), add in a few lines to create, resume, pause, refresh, and destroy TouchHelper, compile before and after the addition (test that the addition actually works on the device, you should see stroke width change after pausing and the screen refreshes), decompile both, and do a diff to see what to add (both the TouchHelper and the SDK itself). You should see, for example, for resume
invoke-virtual {v0, v1}, Lcom/onyx/android/sdk/pen/TouchHelper;->setRawDrawingEnabled(Z)Lcom/onyx/android/sdk/pen/TouchHelper;
Install the APK you want to mod in the emulator in Android Studio and use the debugger to find where in the smali to add TouchHelper.
Finally actually decompile the APK, add in the lines for TouchHelper, add the SDK, and recompile.
Alright, thanks so much! If I have time I’ll try this out!
In the meantime, could you give us a sense of when this feature will see some improvement? I’m not asking for anything astronomical, but this is one of the top issues for me at the moment. If this feature is years off into the future, that’s fine, but I’d like to know it’s years off.
1 Like
Hi, I don’t mean to pester you, really, but this issue is a huge deal to me. It’s a big shortcoming of the technology as a whole; I find my friends on regular tablets having a way easier time writing handwritten notes just because the screen updates at a rapid rate.
I understand the PR issue, but it would mean a lot to me if I at least knew if I should wait up on this feature, or just buy a regular tablet.
1 Like
First of all, I am not related to Onyx, but I still want to explain you, why this is not easily done and will take a little more time or even can’t be resolved.
The point is, that the API of apps like OneNote, Google handwriting input, etc. are made for LCD with at least 30 Hertz. E-Ink screens usually have a frame rate of 5 to 10 fps and since e-ink is static, you can compare its frame rate with the refresh rate of an LCD. Which means, that even slow LCD displays are 4-6 times faster than e-ink. This is why these apps lag. Onyx could try to overwrite the behavior of the Android API for drawing, but this is nothing that can be done easily. Maybe they will never succeed.
The only reliable solution is to contact the developers of the apps and ask for e-ink support and refer them to Onyx API documentation. Since Google even requires fast screens for Play Store certification, you can not expect, that Google would improve its API to work also with e-ink, which indeed would be the best and fastest solution.
2 Likes
I am aware of the differences in hardware between an eink screen and an lcd screen. The solution I wrote down in my post should still work, despite these hardware differences. Unless there are some oversights I missed, I don’t see how an implementation of this wouldn’t at the very least make almost all 3rd party drawing apps usable.
1 Like
Wonderful contribution. I fully add my voice to it.
Please excuse my late reply. I hope you did not misunderstood me.
I am utterly for a real support of 3rd party drawing apps. But in my opinion they should rewrite the Android draw API in the Kernel for a better e-ink display support, instead of implementing something as a “Note mode”. A note mode requires a manual setting that is not necessary and also complicated for some users.
1 Like
Ohhh, I gotcha. TBF, I feel like when I came up with this, it was from the perspective of a user whose only path to extend current functionality is to write user-level code, like apps and ports of apps. As a result, the design I came up with will likely end up being more of a hack than anything else. I mean, if I had the time, I could probably implement this myself as a custom app launcher that you’d install as an apk file.
I think I agree with you, if I were to really have Onyx do something, it’d be to have them rewrite the Draw API (which I think is open source, as a part of AOSP) to kind of automatically use the same ‘quick update’ style for penstrokes near the input of the pen in third party apps. If they did that, they’d basically support all android apps in one fowl swoop. If what @claire said about this being worked on actively is to be believed, that might actually be what they’re doing. It kinda explains why it’s taking so long too, rewriting system libraries is serious business.
I might add an edit or something to the main post, so new people realize this, and realize that better solutions are possible. We should probably make a separate thread on that too tbh, just so this doesn’t get picked up as THE solution to this problem.
1 Like
Thank you for your detailed explanation here. Our relative colleagues are working on this right now. Please be patient.
2 Likes
Hello, Claire, could you share news about progress in solving the problem of delays in drawing with the stylus in 3rd Party Note Apps?
I think the news about this will be interesting to a lot of people.
I was impressed by your video, lectoreNotes was one of my favorite note-taking programs, but it runs very slowly on Onyx Note.
Have you contacted the author of the lectoreNotes program and offered him your own solution to the problem with delays in writing?
I think a lot of people would appreciate it. I am not an android developer and I do not understand what fixes you have made, I would be very grateful if you would describe it in more detail or maybe share your examples somewhere on github.
Yeah no, this is false. What you’re saying is hard is actually very straightforward. They just need to go into the surfaceView creation section, add in a couple lines that create/setLimitRect/openRawDrawing, then setRawDrawingEnabled(true/false) in onPause/onResume (disable touch while we’re at it), for a very simple drawing hook.
My smali hack was entirely done in function overrides, meaning the places I inserted stuff at are already present in AOSP. They literally just need to add in those lines I described.
That was my first time looking at any Android app source/smali, and it only took a few hours to do the above in smali.
The reason I think they’re hesitant about it is because there are a bunch of functions that are very app dependent, such as stroke thickness, erasing (they might just want to do naive erasing, but what the app decides to erase probably will be different, e.g. stroke erasing), smoothing, etc. But you’re not talking about this, because I don’t think what you described is how the Android drawing API works. I don’t think it has anything to do with screen refresh rate.
If an app developer is to implement this, what I described is also most likely what they would do (maybe a souped up version), so there’s very little difference doing this at the app level vs. the system level.
Again, I’m pretty sure this isn’t how the drawing API works.
The “note mode” would literally be what I described in the other post, the only difference being the surfaceView created being the entire screen (probably minus the status bar), rather than just the app drawing area.
Also, you know how little of a difference there is between manual and automatic mode for toggling this? It’s as easy as automatically disabling touch on stylus activity, which I have a (nonpolling) script for. Basically anyone who has written a decent amount of shell scripts can do it.