FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Ignore wrong contest submissions in ProxyService by edomora97 · Pull Request #1142 · cms-dev/cms · GitHub

/ cms Public

Ignore wrong contest submissions in ProxyService - #1142

Merged
andreyv merged 4 commits into
cms-dev:masterfrom
edomora97:fix-ps-multicontest
Dec 17, 2021
ConversationCommits (4)ChecksFiles changed
Merged

Ignore wrong contest submissions in ProxyService#1142
andreyv merged 4 commits into
cms-dev:masterfrom
edomora97:fix-ps-multicontest

Conversation

edomora97 commented Nov 11, 2019 •
edited
Loading

Copy link
Copy Markdown
Contributor

When you are running cms with more than a contest (ie multicontest) ProxyService can be notified by ScoringService of new submissions of any contest, ignoring the -c X flag of PS. This will lead to Inconsistent Data errors in RWS due to submissions of the wrong contest (bad user, bad task, ...). This is especially bad because the wrong submission can be sent with other valid submissions, and RWS drop them all.

The proposed fix just ignores those submissions from ProxyService since ScoringService has no way to know which contest PS is bound to.

Related to #1102 but less intrusive.

This change is 

codecov Bot commented Nov 11, 2019 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #1142 (0327b85) into master (de5a0ca) will increase coverage by 0.04%.
The diff coverage is 22.22%.

@@            Coverage Diff             @@
##           master    #1142      +/-   ##
==========================================
+ Coverage   63.56%   63.61%   +0.04%     
==========================================
  Files         233      233              
  Lines       17117    17126       +9     
==========================================
+ Hits        10881    10894      +13     
+ Misses       6236     6232       -4
Flag Coverage Δ
functionaltests 47.24% <22.22%> (+0.15%) ⬆️
unittests 44.07% <0.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cms/service/ProxyService.py 66.18% <22.22%> (-1.50%) ⬇️
cms/io/rpc.py 93.12% <0.00%> (-1.04%) ⬇️
cms/grading/Job.py 89.28% <0.00%> (-0.90%) ⬇️
cms/service/ResourceService.py 59.11% <0.00%> (-0.89%) ⬇️
cms/grading/Sandbox.py 68.70% <0.00%> (-0.18%) ⬇️
cms/db/base.py 88.11% <0.00%> (ø)
cms/db/filecacher.py 79.14% <0.00%> (+0.30%) ⬆️
cms/service/workerpool.py 68.88% <0.00%> (+0.55%) ⬆️
cms/server/admin/handlers/base.py 69.30% <0.00%> (+0.66%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de5a0ca...0327b85. Read the comment docs.

andreyv requested changes Feb 3, 2021
View reviewed changes

andreyv left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks, conceptually this looks good.

I think it is better placed in submission_scored()/submission_tokened():

  1. _missing_operations() filters by contest id anyway
  2. You can put the code together with the other checks and log a message
  3. dataset_updated() can have its own meaningful message

Could you please update the PR accordingly?

edomora97 commented Feb 4, 2021

Copy link
Copy Markdown
Contributor Author

Sure! I'll take a look again at this in few days, I'm pretty busy right now!

edomora97 commented Dec 11, 2021

Copy link
Copy Markdown
Contributor Author

Sorry for the small delay, I've rebased on the current master and fixed the comments you left.

Not sure why the CI doesn't work, but it seems quite unrelated.

edomora97 mentioned this pull request Dec 11, 2021
Merged
andreyv requested changes Dec 13, 2021
View reviewed changes

andreyv left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks for the update. Please see some comments below.

Comment thread cms/service/ProxyService.py Outdated
Comment thread cms/service/ProxyService.py Outdated
Comment thread cms/service/ProxyService.py
Ignore wrong contest submissions in ProxyService
a0c8be8 
When you are running cms with more than a contest (ie multicontest)
ProxyService can be notified by ScoringService of new submissions of any
contest, ignoring the `-c X` flag of PS. This will lead to `Inconsistent Data`
errors in RWS due to submissions of the wrong contest (bad user, bad
task, ...). This is especially bad because the wrong submission can be sent
with other valid submissions, and RWS drop them all.

The proposed fix just ignores those submissions from ProxyService since
ScoringService has no way to know which contest PS is bound to.
Move the contest_id checks in submission_scored/tokened
fe0d48f
Move PS contest checks before submission validation
326375e 
Filter out the submissions from the wrong contest before rejecting them
if they are hidden/unofficial. Add also some debug logging when it
happens.

This also adds the check in dataset_updated() since AWS may trigger the
same kind of wrong RWS notifications.
andreyv force-pushed the fix-ps-multicontest branch from ee73339 to 326375e Compare December 14, 2021 13:57

andreyv commented Dec 14, 2021

Copy link
Copy Markdown
Member

Thanks, now the changes look good.

Please adjust the code to 79 characters and add periods at the end of log messages. Also, you can print submission id with %d.

edomora97 commented Dec 16, 2021

Copy link
Copy Markdown
Contributor Author

Done!

Btw, is there an automated tool for formatting the code?

andreyv commented Dec 17, 2021

Copy link
Copy Markdown
Member

Thanks, merging.

Well, not currently, but one could be used in the future.

andreyv merged commit bc2a337 into cms-dev:master Dec 17, 2021
stefano-maggiolo pushed a commit to stefano-maggiolo/cms that referenced this pull request Dec 21, 2021
Ignore wrong contest submissions in ProxyService (cms-dev#1142)
3f22b7e 
* Ignore wrong contest submissions in ProxyService

When you are running cms with more than a contest (ie multicontest)
ProxyService can be notified by ScoringService of new submissions of any
contest, ignoring the `-c X` flag of PS. This will lead to `Inconsistent Data`
errors in RWS due to submissions of the wrong contest (bad user, bad
task, ...). This is especially bad because the wrong submission can be sent
with other valid submissions, and RWS drop them all.

The proposed fix just ignores those submissions from ProxyService since
ScoringService has no way to know which contest PS is bound to.

* Move the contest_id checks in submission_scored/tokened

* Move PS contest checks before submission validation

Filter out the submissions from the wrong contest before rejecting them
if they are hidden/unofficial. Add also some debug logging when it
happens.

This also adds the check in dataset_updated() since AWS may trigger the
same kind of wrong RWS notifications.

* Fix line lengths and message format
edomora97 deleted the fix-ps-multicontest branch December 27, 2021 17:43
RezwanArefin01 pushed a commit to RezwanArefin01/cms that referenced this pull request Dec 27, 2021
Ignore wrong contest submissions in ProxyService (cms-dev#1142)
bca4c07 
* Ignore wrong contest submissions in ProxyService

When you are running cms with more than a contest (ie multicontest)
ProxyService can be notified by ScoringService of new submissions of any
contest, ignoring the `-c X` flag of PS. This will lead to `Inconsistent Data`
errors in RWS due to submissions of the wrong contest (bad user, bad
task, ...). This is especially bad because the wrong submission can be sent
with other valid submissions, and RWS drop them all.

The proposed fix just ignores those submissions from ProxyService since
ScoringService has no way to know which contest PS is bound to.

* Move the contest_id checks in submission_scored/tokened

* Move PS contest checks before submission validation

Filter out the submissions from the wrong contest before rejecting them
if they are hidden/unofficial. Add also some debug logging when it
happens.

This also adds the check in dataset_updated() since AWS may trigger the
same kind of wrong RWS notifications.

* Fix line lengths and message format
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

andreyv andreyv approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL