Hey.

Thanks for improving on the documentation... that really helps a lot, compared to few years ago when I had to basically fiddle out most of these things from the code :-)

But one thing:

Why the yields in the collect()? I don’t quite get that.

Shouldn't collect() return sequences of *MetricFamily objects? You now return only a single one, and there even alternating ones like in:
a708480#diff-eadfec94068584be6fec5cce354e065ead34a754bc84ead564adf9e8c497ab10R237-R243

Or maybe you mean this as two independent examples?
What makes it confusing though is (which is however not from your commit):

There collect() alternates between returning (well yielding) a Gauge and Counter (also not as sequences)?

I generally don't quite understand why one would yield here at all.

Isn't the reason for custom collectors to start every time fresh (and get rid of all stale metrics, labels, etc.)?
With yield wouldn't you keep all that?

Thanks,
Chris.

Shouldn't do stuff late night... yield of course makes it an iterator o.O

Still I wonder, is there any advantage over using that in the current form rather than return?

I mean is Python' GC smart enough that it can already clean the objects that have already been yielded and thus give an advantage by reducing e.g. memory use for very large metric sets?

In:

        yield GaugeMetricFamily('my_gauge', 'Help text', value=7)
        c = CounterMetricFamily('my_counter_total', 'Help text', labels=['foo'])
        c.add_metric(['bar'], 1.7)
        c.add_metric(['baz'], 3.8)
        yield c

I could imagine that it may release the GaugeMetricFamily object early... but c? It still has a reference?

In any case it would be nice to have in the docs why generators are used (if it's memory advantages) and how to use them so that this actually works out. :-)

Hi @calestyo, thanks for the read! glad to see you've found the docs useful.

And yes, my bad, the code examples are missing the def collect(self): wrapper which makes it unclear what context they're meant to be in, i'll fix that asap.

Your GC read is right too, yield GaugeMetricFamily(...) inline like that can be reclaimed as soon as the caller moves on, but c keeps a reference in the generator frame until it's done. The main reason for yield here isn't memory though, it's just that the registry iterates over whatever collect() returns and a generator is cleaner than building a list. For most collectors you'd never notice the difference anyway.

I'll open a follow-up PR with both fixes and add a note explaining why generators are used. Thanks again for your feedback.

Shouldn't do stuff late night... - :-)

Hmm, but if you say memory isn't the main reason, the real one should perhaps be given?

And if memory is at least part of the reason, then it might have made sense to explicitly mention that and also explain to people that ideally they should re-use the same identifier so that the GC can kick in.

Even perhaps if they have a case where it's reasonable to first collect a few metrics and only yield all of them afterwards, like in:

for loopOver in SomeJSONelements:
    metric[0].add_metric()
    metric[1].add_metric()
for m in metric:
    yield m

#... next metric again uses `metric` var

I.e. telling that it's still good to use only one identifier metric here, so that the GC can work

Hi Chris, thanks for the response. Good point on both counts! I'll add a note that memory is at least part of the reason for using yield, and mention that reusing the same identifier helps the GC kick in sooner. PR coming shortly :)