This is brilliant, and we should do it at the syntax level so that people can adopt it easily.

However, alt is a subset of a much larger ecosystem that spans a number of concerns, including internationalization, intent, etc. in addition to assistive technology awareness.

Ideally, we'd encourage consistency at the metadata level by adding "blessed" terms to the nbformat JSON schema so that clients aren't playing guessing games. The best of worlds would be to identify an existing standard, whether in JSON schema or whatever: my pick for this would be the web annotation data model which has mappings to w3c annotation-aria which can be accessibly embedded in html.

Practically, this would mean replacing:

"metadata": {"image/png": {"alt": "a picture of an elephant"}}

to...

"metadata": {"image/png": {
    "annotations": [
      # some more verbose JSON that other tools can read and reason about
    ]
  }
}

For some related things around making complex things describable, the p5 folk have done and written about some sterling work.

🛎️ @tonyfast @isabela-pf @manfromjupyter

I honestly do not have much experience in this area and am happy to defer to others for a more formal specification if there is interest in that direction.

I think this is particularly important because while many people have been doing great work to make the tools accessible for the authors of these notebooks this seems like an obvious gap for the readers of these notebooks, especially given the plethora of publishing avenues.

@blois really, thanks for starting this: I don't think anybody is experienced at this point, as we're trying to do something relatively difficult across a large number of pieces.

given the plethora of publishing avenues.

This is definitely the motivation for getting the metadata right. With the right metadata, we can enrich:

• the library experience: as you've started, every kernel completion request can encourage adding this information
• the authoring UI: Lab can have offer tools for annotating stuff, post-hoc, if you don't control the underlying code
• "raw" generated HTML: nbconvert and nbviewer can show annotations
• "soft" publication: sphinx, jupyter-book, etc. can present and make this information searchable
• "hard" publication: more advanced PDF pipelines can make this work (not just browser screenshots)

Starting where you have is a really great place!

@bollwyvl If the desire is more than a plain string then it probably makes sense for this to be specified in nbformat first. I had started here since it appears to be the de facto specification for output metadata fields.

I'd prefer to avoid broadening the scope of this change to to encompass more accessibility features across more types of outputs as I feel that work is much larger, more speculative and lacks a clear owner. Adding an alt field to the metadata is a straightforward fix to the problem.

If there can be a more generalized set of accessibility metadata added to nbformat and outputs then it should be entirely reasonable for those to supersede simpler alt fields similar to how outputs themselves contain multiple prioritized formats.

Please let me know if there are further changes I should make here, otherwise feel free to close this PR if an nbformat specification approach is preferred.

I personally prefer keeping it as:

"metadata": {"image/png": {"alt": "a picture of an elephant"}}

The closer we can keep it to what that coding languages already exists (HTML, JQuery, etc) the better. Users shouldn't have to learn a new language or way of doing things just to support persons with disabilities/handicaps. alt is preferred and if that's already in the product, I say don't change it, but that's just me.

Sorry for the delay in reviewing / merging.

I have no opposition to this, it's just unclear to me if you want to get that in as is, or build consensus first.
I'm happy to get this in master, Ipython 8.0 will not be released soon, so there is still time if this gets in to be refined.

Let me know and I'm happy to merge.

If someone is interested in picking up the work to formally spec accessibility for outputs then I think this PR should be dropped. I opened jupyter/accessibility#37 and attended the regular meeting to discuss, I do not believe anyone currently has the time to pick up the larger work.

I don't think there's any immediate urgency for this issue and am fine waiting to see if we can build traction elsewhere.

The one thing is that once some variant of this lands is that it may be good to try to get matplotlib images to automatically fill in the alt text. I believe the primary work would be somewhere in

At this point I believe that the approach proposed here is the best path forward. I'd like to see this merged so we can move on to encouraging users and libraries to include accessible alt text on images.

I agree that landing this as-is is the best start. It doesn't preclude more detailed abstract approaches, and we shouldn't have to wait for those. This is simple, straightforward, and solves a real issue without introducing any conflict with other ideas.

I'm not speaking from a technical background here, but I would love to see this happen. I've had people in multiple communities now ask me for best practices on how to describe images like this, so I think it this would be a great step in getting a standard and supported method for alt text.

Since this has an approval, what are the next steps to getting this merged?

I believe that the next step is that someone with write access to the repo needs to merge it. Please let me know if there is any action required on my part.

THanks, let's get that in an iterate if necessary.

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

$ git checkout 7.x
$ git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

$ git cherry-pick -m1 59e313b13998ab355470ce8b89c468e3399f07e8

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

$ git commit -am 'Backport PR #12864: Allow specifying alt text for IPython.display.Image.'

4. Push to a named branch :

git push YOURFORK 7.x:auto-backport-of-pr-12864-on-7.x

5. Create a PR against branch 7.x, I would have named this PR:

"Backport PR #12864 on branch 7.x (Allow specifying alt text for IPython.display.Image.)"

And apply the correct labels and milestones.

Congratulation you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove Still Needs Manual Backport label once the PR gets merged.

If these instruction are inaccurate, feel free to suggest an improvement.