UTF-8 support in metric and label names by fedetorres93 · Pull Request #1255 · prometheus/client_java · GitHub

Adds UTF-8 support for metric and label names.

These changes are based on the work done on the Prometheus common libraries here and here

The prometheus-metrics-exposition-formats module will use the new quoting syntax {"foo"} iff the metric does not conform to the legacy name format (foo{})
The prometheus-metrics-model module has a new flag (nameValidationScheme) that determines if validation is done using the legacy or the UTF-8 scheme. This flag can be set via a property in the properties file.
Scrapers can announce via content negotiation that they support UTF-8 names by adding escaping=allow-utf-8 in the Accept header. In cases where UTF-8 is not available, metric providers can be configured to escape names in a few different ways: values (U__ UTF value escaping for perfect round-tripping), underscores (all invalid chars become _), dots (dots become _dot_, _ becomes __, all other values become ___). Escaping has a global default (PrometheusNaming.DEFAULT_ESCAPING_SCHEME) or can also be specified in Accept header with the escaping= term, which can be allow-utf-8 (for UTF-8-compatible), underscores, dots, or values.
This should still be a noop for existing configurations because scrapers will not be passing the escaping key in the Accept header. Existing functionality is maintained.
The prometheus-metrics-exporter-pushgateway module will escape UTF-8 grouping keys in the URL path used when pushing metrics (see Add UTF-8 support in metric and label names pushgateway#689)

Work towards prometheus/prometheus#13095

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Given the ongoing discussion about unit suffixes for OM 2.0 (prometheus/OpenMetrics#286), I think we can take this UTF-8 work as a basis and then add the necessary changes to comply with the final consensus on suffixes.

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Given the ongoing discussion about unit suffixes for OM 2.0 (prometheus/OpenMetrics#286), I think we can take this UTF-8 work as a basis and then add the necessary changes to comply with the final consensus on suffixes.

@fstab are you ok with that?

Update: The client_java maintainers just decided that we'll wait 6 more weeks, until 1 April 2025. If we have OpenMetrics 2.0 by then we will implement that. If OpenMetrics 2.0 is still under discussion in 6 weeks we will merge this PR.

@fstab Good to know, thanks for the update!

Hello @fstab, I just wanted to follow-up on your last comment. Seems like OM 2.0 is still under discussion, so do you think now is a good time to reconsider merging this PR?

We have our client_java community call tomorrow, and can discuss this there. If you have time, feel free to join. See the public Prometheus calendar linked here: https://prometheus.io/community/

@fedetorres93 thanks for the PR!

let me start with some high level questions before an in-depth review:

I think we need a setting for https://prometheus.io/docs/guides/utf8/#otlp-metrics in ExporterOpenTelemetryProperties
the statics in PrometheusNaming should be final - e.g. the scrape handler should pass the escaping as an argument

Signed-off-by: Federico Torres <federico.torres@grafana.com>

@zeitlinger Thanks for the feedback. I made the statics in PrometheusNaming final as you suggested.

About adding a setting in ExporterOpenTelemetryProperties, IIUC that module is translating from Prometheus to OTel format, so it should continue working as it is now. Prometheus' UTF-8 configs affect translations from OTel to Prometheus.

PrometheusNaming.nameEscapingScheme

I can't find that

great - I think I get it now 😄

for formatting, checkstyle issues see CONTRIBUTING.md

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

# Conflicts: # prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/ExporterPushgatewayProperties.java # prometheus-metrics-exporter-pushgateway/src/main/java/io/prometheus/metrics/exporter/pushgateway/PushGateway.java # prometheus-metrics-exposition-formats/src/main/java/io/prometheus/metrics/expositionformats/internal/PrometheusProtobufWriterImpl.java # prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/OpenMetricsTextFormatWriter.java # prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/PrometheusProtobufWriter.java # prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/PrometheusTextFormatWriter.java # prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/TextFormatUtil.java # prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/GaugeSnapshot.java # prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/MetricMetadata.java # prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/UnknownSnapshot.java

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

I found one prometheus.properties file that I think is not needed anymore, otherwise it looks great 🎉

Thanks a lot everyone!

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

fedetorres93 added 3 commits January 3, 2025 12:16

Add UTF-8 support in metric and label names

5445c1d

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Add grouping key escaping for URLs in Pushgateway exporter

7939aba

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Merge branch 'main' into ftorres/utf-8

b5260be

fedetorres93 mentioned this pull request Jan 9, 2025

UTF-8 support in metric and label names #922

Closed

fedetorres93 added 2 commits January 20, 2025 16:26

Merge branch 'main' into ftorres/utf-8

8444ae9

Fix escaping bugs

a593d4e

Signed-off-by: Federico Torres <federico.torres@grafana.com>

fedetorres93 mentioned this pull request Jun 3, 2025

Implement UTF8 Support #916

Closed

fedetorres93 added 5 commits July 8, 2025 15:25

Merge branch 'main' into ftorres/utf-8

5f9726f

Fix tests

f9ce942

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Make statics in PrometheusNaming final

a64dda8

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Fix write errors in tests

afc3156

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Add getter for nameValidationScheme

314838d

Signed-off-by: Federico Torres <federico.torres@grafana.com>

zeitlinger reviewed Aug 4, 2025

View reviewed changes

fedetorres93 added 7 commits August 6, 2025 09:40

Change NameType accessibility

ce9aeb2

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Remove prefix argument from load method in NamingProperties

65bfb6b

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Remove redundant null check in initValidationScheme

e918ca3

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Refactor escaped snapshots creation

e2aacd5

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Extract common logic in escaping tests

7204288

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Parameterize testFindWriter

891d28d

Signed-off-by: Federico Torres <federico.torres@grafana.com>

Add escaping scheme to ExporterPushgatewayProperties

8143f69

Signed-off-by: Federico Torres <federico.torres@grafana.com>

zeitlinger added 25 commits August 14, 2025 14:47

nullaway

0506165

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

is sampler enabled

1b2d2e5

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

c041128

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

8b703d0

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

1c0e85a

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

fb842ab

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

80817b6

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

db0fc2b

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

697effc

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

99fd8f3

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

d9e1e77

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

3662341

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

29c2ba2

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

fd0900c

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

nullaway

5ad2090

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

format

330eb16

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

exclude tests and examples

84c5118

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

docs

ffc8609

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

rename escaping scheme value for no escaping

ae009cf

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

Merge remote-tracking branch 'origin/nullaway' into ftorres/utf-8

3075318

Merge remote-tracking branch 'origin/main' into ftorres/utf-8

74d1a24

format

530d512

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

Merge remote-tracking branch 'origin/main' into ftorres/utf-8

fae2c60

fix

2c53f49

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

fstab approved these changes Aug 18, 2025

View reviewed changes

Comment thread prometheus-metrics-model/src/test/resources/prometheus.properties Outdated

zeitlinger added 2 commits August 18, 2025 15:29

remove unused file

8f04695

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>

Merge branch 'main' into ftorres/utf-8

f9d6e03

zeitlinger enabled auto-merge (squash) August 18, 2025 13:30

zeitlinger merged commit f613ab4 into prometheus:main Aug 18, 2025
7 checks passed

UTF-8 support in metric and label names by fedetorres93 · Pull Request #1255 · prometheus/client_java · GitHub

Uh oh!

UTF-8 support in metric and label names - #1255

UTF-8 support in metric and label names#1255zeitlinger merged 116 commits intoprometheus:mainprometheus/client_java:mainfrom fedetorres93:ftorres/utf-8fedetorres93/client_java:ftorres/utf-8Copy head branch name to clipboard

Conversation

fedetorres93 commented Jan 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fedetorres93 commented Jan 9, 2025

Uh oh!

zeitlinger commented Jan 22, 2025

Uh oh!

fstab commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fedetorres93 commented Feb 14, 2025

Uh oh!

fedetorres93 commented Apr 10, 2025

Uh oh!

fstab commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeitlinger commented Apr 15, 2025

Uh oh!

fedetorres93 commented Jul 25, 2025

Uh oh!

zeitlinger commented Aug 4, 2025

Uh oh!

zeitlinger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zeitlinger commented Aug 4, 2025

Uh oh!

fstab left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

UTF-8 support in metric and label names#1255
zeitlinger merged 116 commits into
prometheus:mainfrom
fedetorres93:ftorres/utf-8

fedetorres93 commented Jan 9, 2025 •
edited
Loading

fstab commented Feb 14, 2025 •
edited
Loading

fstab commented Apr 10, 2025 •
edited
Loading