WIP: Fix font selection by tacaswell · Pull Request #8800 · matplotlib/matplotlib · GitHub

This pulls the commits from #8607 and #8551 together and starts to add starts to add code to standardize extracting the weight and style from the 4 places it could be stashed

Eventually this the weight / style selection code will look something like

fc-list : file > /tmp/font_list.txt

from matplotlib import ft2font
from matplotlib import font_manager
from pathlib import PurePath
with open('/tmp/font_list.txt', 'r') as fin:
    font_paths = [_.strip().strip(':') for _ in fin.readlines()]

fonts = []

for fn in font_paths:
    fn = PurePath(fn)
    if fn.suffix not in ('.otf', '.ttf'):
        continue
    try:
        ft = ft2font.FT2Font(str(fn))
        reasonable_effort_style = [v.lower().split() for v in get_all_style(ft.get_sfnt())]
        reasonable_effort_style += [ft.style_name.lower().split()]
        hit_weight = set(k for v in reasonable_effort_style for k in font_manager.weight_dict
                         if k in v)
        print(ft.family_name, reasonable_effort_style, hit_weight, sep=' | ')
    except RuntimeError:
        print('failed')

attn @cy18 @Rufflewind

looks like you chmod+x'd the file accidentally.

I guess it was supposed to be called "_ttf_header_parse.py" (not tff)

Yes, that would explain why imports were failing on me...

is this always meant to be parsed with Python 3?

It should be both py2 and py3 (but I work in 3 day-to-day). Did I accidentally use a py3 only feature?

the lgtm PR check is flagging some encoding issue. Python 2 uses ascii by default and Python 3 utf-8 so that would explain why that's not an issue for you, but in a 2 environment I think this would be a problem

Try get font information from Platform 3(Windows Unicode) if no information got from Platform 1, encoding 0(Mac Roman)

In things that are fun, I think I found a bug in the metadata in the ubuntu fonts

Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from matplotlib._ttf_header_data import decode_name_table

In [2]: from matplotlib import ft2font

In [3]: ft2 = ft2font.FT2Font('/usr/share/fonts/truetype/ubuntu-font-family/Ubuntu-M.ttf')

In [4]: list(decode_name_table(ft2.get_sfnt()))
Out[4]: 
[(('copyright', 'English'),
  'Copyright 2011 Canonical Ltd.  Licensed under the Ubuntu Font Licence 1.0'),
 (('family_name', 'English'), 'Ubuntu'),
 (('sub_family', 'English'), 'Medium'),
 (('ufi', 'English'), 'DaltonMaagLtd: Ubuntu Medium 0.83'),
 (('full_font_name', 'English'), 'Ubuntu Medium'),
 (('version', 'English'), 'Version 0.83'),
 (('ps_name', 'English'), 'Ubuntu-Medium'),
 (('trademark', 'English'),
  'Ubuntu and Canonical are registered trademarks of Canonical Ltd.'),
 (('manufacturer', 'English'), 'Dalton Maag Ltd'),
 (('designer', 'English'), 'Dalton Maag Ltd'),
 (('description', 'English'),
  "The Ubuntu Font Family are libre fonts funded by Canonical Ltd on behalf of the Ubuntu project. The font design work and technical implementation is being undertaken by Dalton Maag. The typeface is sans-serif, uses OpenType features and is manually hinted for clarity on desktop and mobile computing screens.\r\n\r\nThe scope of the Ubuntu Font Family includes all the languages used by the various Ubuntu users around the world in tune with Ubuntu's philosophy which states that every user should be able to use their software in the language of their choice. The project is ongoing, and we expect the family will be extended to cover many written languages in the coming years."),
 (('url_vendor', 'English'), 'http://www.daltonmaag.com/'),
 (('url_designer', 'English'), 'http://www.daltonmaag.com'),
 (('typographic_family_name', 'English'), 'Ubuntu'),
 (('typographic_sub_family', 'English'), 'Medium'),
 (('compat_ful_font_name', 'English'), 'Ubuntu Medium'),
 (('copyright', 'English (United States)'),
  'Copyright 2011 Canonical Ltd.  Licensed under the Ubuntu Font Licence 1.0'),
 (('family_name', 'English (United States)'), 'Ubuntu Light'),
 (('sub_family', 'English (United States)'), 'Bold'),
 (('ufi', 'English (United States)'), 'DaltonMaagLtd: Ubuntu Medium 0.83'),
 (('full_font_name', 'English (United States)'), 'Ubuntu Medium'),
 (('version', 'English (United States)'), '0.83'),
 (('ps_name', 'English (United States)'), 'Ubuntu-Medium'),
 (('trademark', 'English (United States)'),
  'Ubuntu and Canonical are registered trademarks of Canonical Ltd.'),
 (('manufacturer', 'English (United States)'), 'Dalton Maag Ltd'),
 (('designer', 'English (United States)'), 'Dalton Maag Ltd'),
 (('description', 'English (United States)'),
  "The Ubuntu Font Family are libre fonts funded by Canonical Ltd on behalf of the Ubuntu project. The font design work and technical implementation is being undertaken by Dalton Maag. The typeface is sans-serif, uses OpenType features and is manually hinted for clarity on desktop and mobile computing screens.\r\n\r\nThe scope of the Ubuntu Font Family includes all the languages used by the various Ubuntu users around the world in tune with Ubuntu's philosophy which states that every user should be able to use their software in the language of their choice. The project is ongoing, and we expect the family will be extended to cover many written languages in the coming years."),
 (('url_vendor', 'English (United States)'), 'http://www.daltonmaag.com/'),
 (('url_designer', 'English (United States)'), 'http://www.daltonmaag.com'),
 (('typographic_family_name', 'English (United States)'), 'Ubuntu'),
 (('typographic_sub_family', 'English (United States)'), 'Medium')]

In [5]:

not that the family_name / sub family is inconsistent. One place it is ('Ubuntu', 'Medium') and another it is ('Ubuntu Light', 'Bold'), hence the extra cut-out. Have to sort out where upstream is to report a bug....

- “ -> " - ” -> " - ’ -> ' - — -> - copy-paste from websites is exciting!

FWIW, another case that would be nice to cover is the bold face in the computer modern unicode font (https://ctan.org/pkg/cm-unicode?lang=en, also available as ttf elsewhere online) which has a family name of "CMU Serif Bold Extended Roman"); the line

    weight = next((w for w in weight_dict if sfnt4.find(w) >= 0), None)

in font_manager.py will currently match the font as either weight = 500 (roman) or weight = 700 (bold) depending on the order of iteration on the weight_dict dictionary :-) (on Py3.6+, this means roman (which is incorrect), as it comes first in the dict).

See also https://github.com/anntzer/mplfccache which relies on fontconfig to generate (once) the matplotlib font cache.

Also https://github.com/anntzer/freetypybind/blob/master/lib/freetypybind/query_face.py#L499 for what is I think a mostly complete reimplementation of what fontconfig does, plus support for some common language tags (python ships a bunch of encodings without actually documenting them anywhere afaict (https://github.com/python/cpython/tree/master/Lib/encodings) :-))

Closing as very old and @anntzer has the font config algorithm in python.

cy18 added 2 commits June 25, 2017 23:50

Improve the method of judging weight

30e8248

Added thin and demilight as font weight

ab782ba

tacaswell added status: needs revision topic: text labels Jun 26, 2017

tacaswell added this to the 2.1 (next point release) milestone Jun 26, 2017

anntzer reviewed Jun 26, 2017

View reviewed changes

cy18 and others added 4 commits June 27, 2017 22:13

Look for font information in more locations

a30b491

Try get font information from Platform 3(Windows Unicode) if no information got from Platform 1, encoding 0(Mac Roman)

Recognize more font weights and update mapping

765fd78

Improve font weight guesser

0789e75

Restore original numeric weights

c3ef065

tacaswell force-pushed the fix_font_selection branch from 8651610 to 7a9183f Compare June 28, 2017 02:14

tacaswell added 2 commits June 28, 2017 11:08

ENH: add data for parsing otf / ttf name table

c1fef83

FIX: overhaul how font metadata is extracted

75aa38b

tacaswell force-pushed the fix_font_selection branch from 7a9183f to 75aa38b Compare June 28, 2017 15:09

FIX: remove non-ascii characters

cdcb7f9

- “ -> " - ” -> " - ’ -> ' - — -> - copy-paste from websites is exciting!

tacaswell modified the milestones: 2.2 (next next feature release), 2.1 (next point release) Aug 26, 2017

tacaswell modified the milestones: v2.2, v3.0 Jan 17, 2018

jklymak added the status: needs rebase label May 30, 2018

tacaswell modified the milestones: v3.0, v3.1 Jul 7, 2018

jklymak modified the milestones: v3.1.0, v3.2.0 Feb 7, 2019

anntzer added the topic: text/fonts label Mar 22, 2019

anntzer removed the topic: text label Mar 22, 2019

timhoffm modified the milestones: v3.2.0, v3.3.0 Aug 16, 2019

tacaswell closed this Aug 19, 2019

tacaswell deleted the fix_font_selection branch August 19, 2019 19:53

tacaswell modified the milestones: v3.3.0, unassigned Aug 19, 2019

anntzer mentioned this pull request Jan 13, 2020

Port fontconfig's font weight detection to font_manager. #16203

Merged

6 tasks

WIP: Fix font selection by tacaswell · Pull Request #8800 · matplotlib/matplotlib · GitHub

Repository navigation

WIP: Fix font selection - #8800

WIP: Fix font selection#8800
tacaswell wants to merge 9 commits into
matplotlib:masterfrom
tacaswell:fix_font_selection

tacaswell commented Jun 26, 2017

Uh oh!

anntzer Jun 26, 2017

Uh oh!

anntzer Jun 26, 2017

Uh oh!

tacaswell Jun 26, 2017

Uh oh!

jhelie Jun 28, 2017

Uh oh!

tacaswell Jun 28, 2017

Uh oh!

jhelie Jun 28, 2017

Uh oh!

tacaswell commented Jun 28, 2017

Uh oh!

anntzer commented Jan 2, 2018

Uh oh!

anntzer commented Jan 14, 2018

Uh oh!

anntzer commented Jan 17, 2018

Uh oh!

tacaswell commented Aug 19, 2019

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

		@@ -93,19 +93,26 @@
		'ultra-expanded' : 900}

		@@ -0,0 +1,401 @@

WIP: Fix font selection by tacaswell · Pull Request #8800 · matplotlib/matplotlib · GitHub

Repository navigation

WIP: Fix font selection - #8800

WIP: Fix font selection#8800tacaswell wants to merge 9 commits intomatplotlib:mastermatplotlib/matplotlib:masterfrom tacaswell:fix_font_selectiontacaswell/matplotlib:fix_font_selectionCopy head branch name to clipboard

Conversation

tacaswell commented Jun 26, 2017

Uh oh!

anntzer Jun 26, 2017

Choose a reason for hiding this comment

Uh oh!

anntzer Jun 26, 2017

Choose a reason for hiding this comment

Uh oh!

tacaswell Jun 26, 2017

Choose a reason for hiding this comment

Uh oh!

jhelie Jun 28, 2017

Choose a reason for hiding this comment

Uh oh!

tacaswell Jun 28, 2017

Choose a reason for hiding this comment

Uh oh!

jhelie Jun 28, 2017

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Jun 28, 2017

Uh oh!

anntzer commented Jan 2, 2018

Uh oh!

anntzer commented Jan 14, 2018

Uh oh!

anntzer commented Jan 17, 2018

Uh oh!

tacaswell commented Aug 19, 2019

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

WIP: Fix font selection#8800
tacaswell wants to merge 9 commits into
matplotlib:masterfrom
tacaswell:fix_font_selection