| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
ok, i tested it with on Manjaro (py 3.7.2) and on Centos6(py 2.6.6), and it appears to be working correctly |
Sorry, something went wrong.
|
Hey @Ferroin if you could find a time and test the PR a bit it would be very nice |
Sorry, something went wrong.
|
ok, i don't think we need to run a job if job.check() returns False in main. It can be error prone, instead we can treat all jobs in main as autodetection jobs, so if a job's check(), for some reason, succeed in child process and failed in main we will retry check again and again |
Sorry, something went wrong.
|
@ktsaou if you are ok with the changes this is ready. |
Sorry, something went wrong.
|
ok, here we go, i am on a high alert ⏰ |
Sorry, something went wrong.
…tdata#5552) ##### Summary This PR adds (major) changes only to `python.d.plugin` file. Fixes: netdata#5525 `pyhton.d.plugin` imports a lot of additional packages during initial module initialization/job creating/checking and there is no way to unimport them, even if they arn't needed. It consumes relatively a lot of ram. ___ Memory utilization comparing before/after the PR (one job `example` module, py3.7.2): > 21.1 => 8.8 MiB  ##### Component Name [`collectors/python.d.plugin`](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/python.d.plugin.in) ##### Additional Information This PR adds separate process for initial module checking. Logic: - main process spawns checker process - checker process loads every module, loads module config, creates jobs and runs job.check() for every job, if check success it adds the job to the list. - checker process returns list of modules and jobs. - main process loads only active modules, etc.
| Back | FazBrowse Home | New Git URL |
Summary
This PR adds (major) changes only to python.d.plugin file.
Fixes: #5525
pyhton.d.plugin imports a lot of additional packages during initial module initialization/job creating/checking and there is no way to unimport them, even if they arn't needed. It consumes relatively a lot of ram.
Memory utilization comparing before/after the PR (one job example module, py3.7.2):
Component Name
collectors/python.d.plugin
Additional Information
This PR adds separate process for initial module checking.
Logic: