Updated! Updates are shown in quote text like this. Some scores are updated following app updates.

An Apps Experiment

Cross-posted from https://lemmy.world/post/18159531

Introduction

This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I’ve seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.

Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.

How I did it

I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.

I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.

I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @jordanlund@lemmy.world – Thanks, Jordan! This made for 20 apps that were tested.

Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @marvin@sffa.community, which was posted about a year ago in !meta@sffa.community (here).

I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.

Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.

In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.

Results

Out of a possible perfect 10, 6 apps displayed all markdown correctly:

Alexandrite - 10.0

Connect - 10.0

Jerboa (Official Android client) - 10.0

Photon - 10.0

Summit - 10.0

Voyager - 10.0

Quiblr - 9.5

Arctic - 9.3

Interstellar - 9.1

Lemmuy-UI - 9.0

Thunder - 8.9

Tesseract - 8.6

mlmym - 8.0

Racoon - 7.6

Boost - 7.3

Eternity - 7.0

Lemmios - 6.9

Sync - 6.9

Lemmynade - 6.1

Avelon - 5.7

More details of testing here

Disclaimers

Disclaimers

I Love Lemmy Apps (and their devs)

Lemmy apps devs work very hard, and invest a lot in the platform. Lemmy is better because they are doing the work that they do. Like, a LOT better. Everyone who uses the platform has to access it through one app or another. Apps are the face of the entire platform. Whether an app is a FOSS passion project, underwritten by a grant, or generating income through sales or ads, no one is getting rich by making their app. It is for the benefit of the community.

This is not meant to be a rating of the quality or functionality of any app. An app may have a high rating here but be missing other features that users want, or users may love an app that has a lower rating. This is just about how well apps handle markdown.

This is pretty unscientific

You’ll see my methodology above. I’m not a scientist. There is probably a much better way to do this, and I probably have biases in terms of how I went about it. I think it’s interesting and probably has some valuable information. If you think it’s interesting, let me know. If you think of a better way, PM me and I’d be happy to share what I have so you don’t have to start from scratch.

My only goal is to help the community

I do think that accurately displaying markdown should be a standard expectation of a finished app. I hope that devs use this as an opportunity to shore up the areas that are lagging, and that they have a set of standards to aim for.

I don’t have any Apple things

Sorry. This is just Android and Web review. If someone would like to see how iOS apps are doing, please reach out and I’ll share how we can work together to include them.

    • stormio@lemmy.caEnglish
      0·
      6 months ago

      I wasn’t sure if Lemmuy-UI in the results list was a typo or an alternative interface. Now I know. 😄

      • gedaliyah@lemmy.worldOPEnglish
        0·
        6 months ago

        In doing this I learned that there are “correct” but also “preferred” ways to use markdown. A heading should have a space after the # even though it is correct either way.

        ##Heading

        Heading

        These lines may be the same or different in different apps.

        • gedaliyah@lemmy.worldOPEnglish
          0·
          6 months ago

          I know that it works on some sites (reddit for example). Generally, it is not preferred.

          • maegul (he/they)@lemmy.mlEnglish
            0·
            6 months ago

            Didn’t know it worked on reddit. Generally it seems necessary to require the space as it disambiguates headings from hashtags, and also makes the raw text more readable.

        • JackbyDev@programming.devEnglish
          0·
          6 months ago

          Compare the source of your comment to the one you’re replying to. Those are two different things. I’d argue it’s a workaround of anything.

        • gedaliyah@lemmy.worldOPEnglish
          0·
          6 months ago

          Yes, I’m not sure if that is meant to be a placeholder or a substitute for native user links. What it actually does is generate markup that converts the username into a web link, which is fine for most circumstances, but not ideal. A plaintext username should automatically link to the user. This creates an inconsistent behavior between posts depending on where (and when) they were typed.

          In other words, it’s a very helpful feature, but it is not recognizing and linking usernames.

            • BentiGorlich@thebrainbin.org
              0·
              6 months ago

              yeah exactly. On mbin it works this way and lemmy inserting the link breaks that. But it does it for communities in the community description sometime as well, though I don’t know if it is just a user “error” or a lemmy error

          • BentiGorlich@thebrainbin.org
            0·
            6 months ago

            Actually that behaviour is very annoying to other platforms. Mbin for example can only link to the lemmy server this user is on and no longer the local profile of that user. Example: @ user @ lemmy.instance gets converted to [@ user @ lemmy.instance](https:// lemmy.instance/u/user so on mbin this does not open the profile of the user on the local server, but instead links the lemmy instance, so you leave your instance to view the profile.

            (spaces included so this won’t get converted to mentions, etc)

              • BentiGorlich@thebrainbin.org
                0·
                6 months ago

                Yes they are, but you have my profile on your server and you do not need to leave the server to view my profile… @ user @ lemmy.instance should link to https:// mbin.instance/u/@user@lemmy.instance and not to https:// lemmy.instance/u/user

                • JackbyDev@programming.devEnglish
                  0·
                  6 months ago

                  I’ll have to try on desktop, in the app it isn’t very clear what exactly it’s looking at to see profiles.

        • threelonmusketeers@sh.itjust.worksEnglish
          0·
          6 months ago

          What happens if you press tab or click on the suggested item at the point in your screenshot?

          For me, it inserts the link at the cursor position, but doesn’t replace the bit you’ve already typed, resulting in @gedal[@gedaliyah@lemmy.world](URL).

          Anyone else have this issue?

          • Nothing4You@programming.devEnglish
            0·
            6 months ago

            on firefox, if i type @gedal and click or press tab once it replaces the text with [@gedaliyah@lemmy.world](https://lemmy.world/u/gedaliyah) . the behavior is the same whether i hit tab, enter or click the text.

            • threelonmusketeers@sh.itjust.worksEnglish
              0·
              6 months ago

              if i type @gedal and click or press tab once it replaces the text with [@gedaliyah@lemmy.world](https://lemmy.world/u/gedaliyah)

              Ah, you are correct. It turns out that the issue I was encountering was a little more subtle.

              If I type all the way to @gedaliyah@ and click or press tab once it replaces only the second ‘@’, resulting in @gedaliyah[@gedaliyah@lemmy.world](https://lemmy.world/u/gedaliyah).

  • pe1uca@lemmy.pe1uca.devEnglish
    0·
    6 months ago

    I’m looking at this in eternity and seems only spoilers don’t work from the post you linked.
    User and community links work properly.

    • gedaliyah@lemmy.worldOPEnglish
      0·
      6 months ago

      I’m very excited to see a new update from eternity. It’s always been a solid app and is likely to be my daily driver again. Right now I’m using raccoon mostly, although I was pretty disappointed to see the low score on markdown

      • squirrel@discuss.tchncs.deEnglish
        0·
        6 months ago

        Same here, switched from Eternity to Raccoon. Maybe Raccoon’s @DieguiTux8623@feddit.it can help with better markdown support.

        • DieguiTux8623@feddit.itEnglish
          0·
          6 months ago

          We were working on this but it’s a hard task and Compose Multiplatform is still not mature as a technology, there are few libraries and those which exist do not work well.

          Help would be welcome, otherwise people can use Jerboa or Voyager, nobody is forcing anybody to use any app (especially if they are FOSS).

          • squirrel@discuss.tchncs.deEnglish
            0·
            6 months ago

            I can imagine it’s a hard task. I read from OP’s post that most apps struggle with it, so it must be hard.

            Sorry, I didn’t mean to put you on the spot like that. Just wanted to ping you. I would gladly help, but sadly I lack the skills to do so.

            • DieguiTux8623@feddit.itEnglish
              0·
              6 months ago

              Don’t worry @squirrel@discuss.tchncs.de you have been one of the ones who helped the most and I am glad we had the opportunity to collaborate. I don’t appreciate the original poster though, because it only gave general hints of what has been tested but no evidence about the tests. I know that some features (like subscripts, superscripts, strikethrough, tables, footnotes) are not supported BUT the perfect Markdown rendering has never been a priority for me.

                • DieguiTux8623@feddit.itEnglish
                  0·
                  6 months ago

                  The app has a very small user base of approximately a hundred people worldwide and I know almost all of them one by one, but I’ve never heard of you. I’m glad to meet you, even in these unfortunate circumstances.

                  By the way I am considering closing the project, because I am being personally damaged by what is happening now, and my reputation as a professional has been compromised.

  • Gointhefridge@lemm.eeEnglish
    0·
    6 months ago

    I’ve been using Mlem since week 2 and I have no idea if I’m missing anything or not. I’ve never visited any instance on anything but Mlem.

  • maegul (he/they)@lemmy.mlEnglish
    0·
    6 months ago

    So what’s the technological story here? I’m guessing lemmy itself uses a particular markdown parser that could probably be extracted and used in other contexts as it’s likely written in rust and should therefore be pretty portable without too much effort.

    Are other apps just using whatever markdown parser is convenient to them? Is this something that the lemmy and threadiverse community could converge on? Even the fediverse as a whole where just about every platform other than mastodon supports writing in some for of markdown … feels like a pandoc like utility could go far.

    • gedaliyah@lemmy.worldOPEnglish
      0·
      6 months ago

      I’m probably not the person to ask, to be honest. Lemmy as I understand it is the protocol that exchanges the information about posts, etc. The post content is stored and shared as plaintext, but Lemmy also has instructions about how a UI should interpret the text and serve it to the user.

      Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.

      • maegul (he/they)@lemmy.mlEnglish
        0·
        6 months ago

        Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.

        Oh yea styling isn’t the issue here … it’s whether the markdown is correctly interpreted and rendered. AFAIU, lemmy doesn’t have any instructions about how to interpret the text, just some standard that they’ve chosen to use, along with their open source software for doing so (as they’ve built too clients, the default web UI and Jerboa).

  • yo_scottie_oh@lemmy.mlEnglish
    0·
    6 months ago

    cross-posted

    Minor nit pick, but did you know that Lemmy has actual cross posting functionality?

    Either way, interesting study. This is the type of content that I Red er… Lemmy for, so thanks for posting. I use Voyager myself, being an Apollo refugee.

    • WanderingVentra@lemm.eeEnglish
      0·
      6 months ago

      Ya I’ve only been able to cross post on the web UI. I’ve seen apps like Jerboa and Voyager at least show cross posting correctly, I just wish they made it easier to cross-post in app.

      • micahmo@programming.devEnglish
        0·
        6 months ago

        Thunder allows cross-posting! It should follow the web UI implementation (where the body of the new post has a link to the original, plus the original contents in a quote block).

        • WanderingVentra@lemm.eeEnglish
          0·
          6 months ago

          Oh nice! That’s the one I’ve probably tried the least so I’ll have to give that one another go. (Not for any particular reason, I just got used to the UI of the first couple other apps I tried). Thanks for the good news!

      • Microw@lemm.eeEnglish
        0·
        6 months ago

        Boost has a very easy-to-handle implementation of crossposting

    • gedaliyah@lemmy.worldOPEnglish
      0·
      6 months ago

      Yeah, cross posting is another quirky Lemmy thing. AFAIK it just generates a new post with the same content, and also maybe varies by app. That could be wrong though: I’m not sure.

      • threelonmusketeers@sh.itjust.worksEnglish
        0·
        6 months ago

        AFAIK it just generates a new post with the same content

        Yup, that is exactly what it does. So if the original post is edited, none of the changes propagate to any of the crossposts.

        In my opinion, crossposts should embed the original post, not simply copy a snapshot of the content at the time the crosspost is made. That’s a Lemmy issue though, not an app issue.

  • Eggyhead@kbin.run
    0·
    6 months ago

    It’s been a month since I’ve been able to post anything from my lemmy.world account using any app.

    • jordanlund@lemmy.worldEnglish
      0·
      6 months ago

      Weird, I use Boost all the time. Did you turn on 2FA or something? Maybe try removing and re-adding your account?

  • Squorlple@lemmy.worldEnglish
    0·
    6 months ago

    There are a lot of image/gif(?) posts that I haven’t been able to view either on the Memmy (Apple) app or in-browser with either Safari (Apple) or Google Chrome. I imagine it comes down to the file types as well as the lack of native hosting to standardize posts of different media types, but I’m not the techiest person to consult on that. One downside of the fediverse is the lack of standards for file hosting/conversion/displaying to ensure that all media can be accessed regardless of the browser/app (or, alternatively, the lack of an all-encompassing app for all devices [Jerboa sounds like the closest to this to me but it is not available for iOS yet]), as well as the self-funded nature of the instances commonly not having the budget to natively host multimedia content such as videos.

  • jordanlund@lemmy.worldEnglish
    0·
    6 months ago

    @gedaliyah@lemmy.world iOS testing, not sure how you score these so I just listed out the broken stuff.

    Arctic - Link opens in App. Headings fail, images fail, everything else looks fine.

    Avelon - Link opens in browser, not app. Manually went to test post. Bold+Italic fails (Italic works, not Bold). Table fails. Horizontal Rule fails. Spoiler fails. Everything else looks good.

    Bean - Link opens in browser, not app. Manually went to test post. Text formatting block fails so hard, it’s not even visible(!) Heading fails. Code Block fails, Inline Code fails. Links and Image work, but not inline, only at the bottom of the post. Table fails. Hotizontal rule fails.

    CheeseBot - Did not test. $2.99, no free version.

    Lemmios - Link opens in app. Everything looks and works great EXCEPT Spoilers.

    Mlem - Link opens in browser, not app. Manually went to test post. As with Lemmios, everything looks and works great EXCEPT spoilers.

    Remmel - Instant fail. No development in 2 years, unable to even add an instance or an account. Non-starter.

    Thunder - Hard to test. Lots of lag for some reason. Link opens in browser, not app. Manually went to test post. That being said, EVERYTHING worked. The lag may have been because I had just linked my account. Testing everything above, then coming back to Thunder, I found it fast and responsive.

    Voyager - Link opens in app. EVERYTHING worked. No notes.

  • SmokeInFog@midwest.socialEnglish
    0·
    6 months ago

    If you’re getting that granular then you must’ve had to record the data somewhere. Did I miss where the OP is sharing their data set?