Update Labour to get specific properties when getting person - #1829
tamuri wants to merge 12 commits into
Conversation
Sorry, something went wrong.
tbhallett
left a comment
There was a problem hiding this comment.
This makes good sense to me.
Sorry, something went wrong.
- the decorator allows us to list none or more properties needed to evaluate the function - does not change the logic inside the function, but is a bit of metadata that can be used by callers
|
Trying a different approach to speed this up further. This is has reduced runtime of predict from 4% to <0.3% A decorator person_properties in labour_lm adds metadata to the function specifying exactly which person properties from the population dataframe (if any) are needed to evaluate the function. If nothing it needed, it is set to an empty list. The labour module uses this metadata to grab only those specified properties from the population dataframe. Sometimes the required modules are not available and we have to remove their properties from the list. Sorry, @tbhallett @joehcollins, do you mind looking at this again. |
Sorry, something went wrong.
… then new simulations in the same process may fail.
Profiling finds that repeated df.loc[[person_id]], where all properties are retrieved, is a bottleneck in the Labour module. This PR updates the module to only retrieve those properties necessary.
Joe - although each labour linear model only uses a few of these properties, grabbing everything necessary for all of them is an easy change for now.
I'm running profiling again to see if further improvements can be made.