Synthesia Talk

Tried to use Synthesia Meta Editor to get my saved fingering for Android Tablet. I put the .synthesia file in Downloads or Music folder in android the same as Midi file and tried it multiple times, but it doesn't seems to work or be in sync. On the other hands, doing the exact same thing using my iPad gets a good result.

Is it a bug or did I do something wrong?

Is it an Android 11 or later device?

I suspect this is being caused by a new "security" measure from Google. In Android 10 they took away the ability for an app to use the normal "let me get a list of all the files in folder X" functions but there was a provision where you could request "wait, we haven't had a chance to migrate to your convoluted new sort-of-database-like system yet, so give me the old behavior for now". When Android 10 was released, they told app developers that workaround was only going to last until Android 11 so we'd better act fast. But when the feedback started pouring in from every Android developer that their new system was awful and that it was better the way it used to work, Google compromised and let the "old" function calls (the ones that work on every other platform, including iOS) work again, but the list of files returned would mysteriously and automatically be filtered down to a known set of "media" file types. Luckily for Synthesia, MIDI files were on that list. Unluckily for Synthesia: .musicxml, .mxl, .synthesia, and .sf2 SoundFont files are not on that list.

So, from Synthesia's perspective, Android will never acknowledge the existence of metadata files to the app, even though they're sitting right there.

To my knowledge, there is no way to request a custom extension to the list of visible/allowed file types. The only workaround is to have your app use the "Request File System Access" privilege, which switches back to the pre-Android 11 behavior where all files are visible again. I enabled it for Synthesia and every app submission after that was automatically denied in under 1 minute until I disabled it again. The denial reason mentioned an appeal form, but of course it's a dead link. And even if they did allow it, the permission isn't granted automatically. The user has to manually tap through a scary-sounding dialog box in the app to be taken to the Settings app, where there's a giant red warning telling you not to grant the permission unless you're really sure.

The whole thing is a frustrating mess. So right now Android 11 and later can't use metadata files or even load MusicXML songs. Thanks Google.

I don't have a good answer or any sort of workaround. I was thinking of enabling the permission in the manually-downloadable-directly-from-our-website APK version of the Android app, but that will take some build system changes on my side and it requires another scary prompt on the user side about unlocking app sideloading (along with all the extra effort of manually downloading the thing outside of the Play Store).

My hope is that someone over there notices what a disaster this situation is and just rolls the whole thing back at some point (the same way they backpedaled on the first revision). Then the workaround will just be "don't use Android versions between 11 and xx". But I haven't heard anything like this yet.

Even if we tried to switch to their new system--which would be a lot of extra effort for just one platform--it boils down to the user manually choosing a file from a file picker dialog box. This paradigm completely breaks Synthesia's file scanning and song list idea. You'd have to manually pick a song from a picker and then manually pick a .synthesia file that may or may not have associated metadata in it. It would be very strange. And only on Android. Because they said so.

Hi Nicholas,

Thanks for your lightning fast and effortful response. I was using Android 12 to test it out, so was a bit hopeless as of yesterday.

Today I tried again but noticed that my Android tablet was still able to read MusicMXL file just fine, so I was still hopeful that it is not stupid Google who blocks wonderful app developers from accessing necessary files. MusicXML isn't that popular filetype. I used a whole different song and tried the steps from the beginning, and to my surprise, it worked! I guess in the first try it didn't work because trying over and over again is fruitless, as somehow Synthesia remembered the original metadata from the first file and tried to use the localized version of .synthesia first. Your Synthesia still works perfectly fine in the (near) latest Android !

The steps of trying to figuring out things work is quite confusing though, I hope that there is some kind of display message in the app to notice which .synthesia are being used for which song, like a debugger would cause us less headache, but I guess it's not a very popular request. Thanks for supporting

tdngo wrote: ↑11-11-22 10:01 pmToday I tried again but noticed that my Android tablet was still able to read MusicMXL file just fine...

This actually makes me more nervous than anything. I hadn't heard those security measures were optional. Maybe your tablet vendor has them turned off? How strange.

tdngo wrote: ↑11-11-22 10:01 pmThe steps of trying to figuring out things work is quite confusing though...

I agree, sorry. There are several places Synthesia can gather metadata. Here's the list in order of priority (lowest to highest):

- Metadata built into the app itself (for, say, the built-in songs or songs from our store).
- Metadata from .synthesia files.
- Metadata entered manually inside the app, which overrides everything else.

There isn't any way to tell which one you're getting. And there isn't any way to reset or ignore one of the levels outside of. So it's very easy to change a single finger hint inside the app, which will then ignore any .synthesia metadata changes forever. Continuing with the example for finger hints: the only ways to step back down a metadata level are to get rid of that manual customization (by either deleting fingerHints.xml in the data folder or manually setting the hints to exactly match those found in the .synthesia file. Then Synthesia will stop believing you have local changes.

It's kind of a mess, really. It'd be nice to have a way to pick between them. Like a "profile" list or something like that, where you could toggle quickly between different sets of hints and hand parts.

Hello,

I discovered this thread a bit too late.

Indeed I have just bought an android tablet to replace an Ipad whose battery is faulty and which I use exclusively for Synthesia.
I have around 400 midi songs customized with .synthesia files and wallpapers.
I was already saddened to lose my progress but there is a cold shower.

If you have a solution, even a little technical, I'm interested!

I would like as much as possible to avoid buying a new Ipad, it's really painful to have to go through Itunes and anyway my banker doesn't agree.

Tchernosplif wrote: ↑01-04-23 4:47 pm... it's really painful to have to go through iTunes...

This might be more salt in the wounds, but since a couple years ago, the Files app on the iPad makes it a lot more convenient/faster than the old iTunes way of managing files. You can copy (MIDI and .synthesia) files around in bulk, organize them into folders, etc. all right on the iPad itself without a computer connection.

Nicholas wrote: ↑11-11-22 12:08 am I was thinking of enabling the permission in the manually-downloadable-directly-from-our-website APK version of the Android app, but that will take some build system changes on my side

Hello Nicholas,
It would be really great if you could provide a modified apk, even if it's not the very latest version.

I have installed a previous version of the app 10.7.5600 and here it works again

Older versions are still able to escape the new security "feature" in Android 11 because the (older) app targets an earlier versions of Android. In those cases, Android tries not to pull the rug out from an app that might not be expecting the new behavior. (I vaguely recall reading that even this loophole will be closed in newer versions of Android, so this is probably a time-limited work-around.)

I wish it were as easy to simply keep targeting an older version, but keeping your app updates targeting the latest version of Android is another one of Google's requirements. So our hands are tied pretty tightly here.

Maybe you can make an import of sorts in the app that can import .synthesia file

matthacker wrote: ↑01-09-23 9:22 am I have installed a previous version of the app 10.7.5600 and here it works again

Great, this version works on my tablet (Android 12); I find folders and metadata fine. Thank you matthacker for the tip!

I have disabled Play Store automatic updates and I will try to avoid inadvertently updating the app...maybe there is a way to block an app's updates!?

matthacker wrote: ↑01-09-23 3:10 pmMaybe you can make an import of sorts in the app that can import .synthesia file

I'm going to do this (in the very short term). The workaround will just be new (Android-only) buttons below the SoundFont list and on the Songs tab under Settings that will let you add these files using Android's new file picker interface. I believe it'll only be able to add files one by one, but that's better than not being able to do it at all.

This is going to be part of a small bug fix release that is still labeled 10.9 and it's the top item on the task list.

Nicholas wrote: ↑01-13-23 1:47 amThe workaround will just be new (Android-only) buttons below the SoundFont list and on the Songs tab under Settings that will let you add these files using Android's new file picker interface. I believe it'll only be able to add files one by one, but that's better than not being able to do it at all.

Will individual files added in this way persist after completely closing and re-opening the app?

I don't know the details about Android versions and security permissions, but I have encountered a possibly similar issue:

• I wanted to use a simple offline JavaScript HTML page I wrote in the web browser of my Android phone. I wanted to add it as a bookmark to the home screen to easily open it.
  
  I made the HTML page on my computer copied it to my Android phone. I found the HTML file in the file browser app on my phone and opened the file in the web browser. However, after I completely closed the web browser (by going to the recent apps screen and closing it from there), then re-opened the web browser, the link to the HTML file was no longer valid. It seems the Android operating system creates a randomized link for the file that only persists until the app is closed.

If this same kind of restriction applies to the file picker workaround you want to add to Android version of Synthesia, I guess you'll have to import a copy of each selected file to some private app storage area. If the user needs to update a .synthesia file or sound font file, they'll have to re-import it. If the user wants to forget an imported file, I guess you'll need another button or screen to list the previously imported files and have an option to remove them.

Bavi_H wrote: ↑01-14-23 2:30 am... I guess you'll have to import a copy of each selected file to some private app storage area. If the user needs to update a .synthesia file or sound font file, they'll have to re-import it. If the user wants to forget an imported file, I guess you'll need another button or screen to list the previously imported files and have an option to remove them.

It's a mess, isn't it?

Completely unnecessary work imposed by a giant megacorp just to keep basic functionality working...

Another idea you might explore is stuffing the data currently stored in ".synthesia" files into a Sequencer-Specific Meta Event in a MIDI file, so the new Android versions of Synthesia could access the ".synthesia" data by finding it in MIDI files it still has permission to access.

(It could still be a sidecar file, perhaps with something like .synthesia.mid as the file suffix? Or perhaps you could allow storing song-level Synthesia metadata inside of the MIDI file it applies to, however, that might not work with the existing hash methods Synthesia uses?)

(I saw your other post that you linked here before this one.) This is one of those "thanks, I hate it" kind of workarounds that would actually solve the problem but would make me feel awful the entire time I spent implementing it.

It's certainly a creative solution. I'll have to think about how I feel about it. I do like that we're still not modifying the MIDI files themselves. Maybe everything should just tack a .mid extension onto the end and Synthesia could just inspect the filename, stripping off spurious ".mid"s until it rediscovers what it was supposed to be. That would work for SoundFonts, too.

A small update with nitty gritty programming details:

So far the workaround mentioned a few posts up is going according to plan. You can see every kind of file in Android's native file-picker and there's even a way to allow the user to be able to do a multi-select.

The "path" that comes back from the file picker is this strange "content://..." URI that you can't do a whole lot with in that form. But, you can call takePersistableUriPermission which will make it continue to appear to your app even across device reboots (supposedly). So I am happy there won't be any temporary-copy shenanigans. Synthesia can use the files in-place.

Even getting the file's original name (to, say, show in the song list or title bar) from the content URI is a bit of an ordeal, but it's doable.

The "interesting" part is that you can't just open the file like a normal file (using C's fopen or C++'s fstream or whatever). You are allowed exactly one way to access it: getting a (Java) FileDescriptor for it, which translates to a super low-level Unix FD handle. Luckily, Ian already has support for loading SoundFonts from file descriptors into BASSMIDI. And I'm going to tinker with mmap for the first time to see if I can't get them to look like an in-memory file, which I already have code to support. (If that goes smoothly, I may eventually migrate more of Synthesia's loading code to mmap to squeeze out some of the slowness in the standard library's IO routines.)

The only gross part is that all of this requires lots of extra tendrils to be added between a handful of layers so that the app can keep track of things like "is this a normal path to a SoundFont or one of these strange content URIs that needs to be opened/handled differently?" all over the place. But we're getting there.

I've also fixed three or four crashes and other bugs in the meantime. Those went out to the other platforms already. As soon as this Android stuff is finished, I'll push another 10.9 update

I just pushed an Android beta that has the first half of this finished: SoundFonts can be added again. (It's still being reviewed by Google Play, but the direct .apk link in the pink box on the dev updates forum should work.)

On Android 11+ the SoundFont list is now always shown on the "Settings --> Built-in MIDI Synthesizer" screen. There is now a "+" button below the list that will show the system file-picker. It has multi-select enabled, so you can find and add all your SoundFonts at once.

(This beta also includes a handful of other crash fixes that have trickled out to the other platforms over the last couple months.)

Like I'd hoped, I was able to find a solution that doesn't involve any copying-around of files. They're read directly from the file-system, in place.

Next up is getting the same thing to work for .synthesia and .musicxml files. The back-end, infrastructure'y bits are finished, but there are still lots of tendrils everywhere because fifteen years ago I made the assumption that file paths would continue to exist.

This should finally be done now (in 10.9.5914). Sorry it took so long.

There is a new "+ Add" button at the lower-left corner of the song list screen (only on Android 11 and later). That will show the system's file browser. You can multi-select and it works with MIDI, MusicXML, and Synthesia metadata files.

As a bonus, there are a couple MIDI device handling fixes included: the USB-MIDI driver works again on Android 12+ and the Android M driver can now send instrument change messages that aren't just for grand piano.

Thanks for the bugfix regarding the add button on Android. Unfortunately there is a little bug. I tried it on Samsung Galaxy Tab A8. After selecting the Musicxml file the app immediately crashes. After that the app cannot be started anymore. It crashes on every start.
Issue can only be solved by uninstalling and installing the app again.
The issue is reproducible. I tried it again with another musicxml file. Same behaviour.

Kind regards
André