FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

TransactionWrite sanitizes items by ckhsponge · Pull Request #897 · Dynamoid/dynamoid · GitHub

TransactionWrite sanitizes items - #897

Merged
andrykonchin merged 10 commits into
Dynamoid:masterfrom
ckhsponge:ckh/transaction-sanitize-item
Sep 6, 2025
Merged

TransactionWrite sanitizes items#897
andrykonchin merged 10 commits into
Dynamoid:masterfrom
ckhsponge:ckh/transaction-sanitize-item

Conversation

ckhsponge commented Apr 18, 2025
edited by andrykonchin
Loading

Copy link
Copy Markdown
Contributor

TransactionWrite should call sanitize_item so that nils are not persisted. Otherwise GSI will fail with a nil range key.

Closes #885

codecov Bot commented Apr 18, 2025

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.88%. Comparing base (37572d7) to head (20d2eea).
Report is 27 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #897      +/-   ##
==========================================
+ Coverage   91.56%   91.88%   +0.31%     
==========================================
  Files          75       76       +1     
  Lines        3700     3856     +156     
==========================================
+ Hits         3388     3543     +155     
- Misses        312      313       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

expect(doc).to be_persisted
end
end
end

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It's kind of integration tests, so it makes sense to have them, but it will be great to have more thorough specs for each transactional method, like it's done for non-transactional ones:

https://github.com/Dynamoid/dynamoid/blob/master/spec/dynamoid/persistence_spec.rb#L2996-L3037

Such specs would test:

  • what actual value of a nullified attribute is persisted (it either has null value or is removed in the item)
  • and how store_attribute_with_nil_value option affects this behaviour

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm thinking to not add tests for store_attribute_with_nil_value = true when there is an index. This fails even without transactions. I suppose I could add a test that a ValidationException is thrown in this case. And that test should exist for both transactions and non-transactions.

  let(:klass_with_gsi) do
    new_class(class_name: 'Ferret') do
      field :name
      field :age, :number
      global_secondary_index hash_key: :name, range_key: :age, projected_attributes: :all, name: 'str_num'
    end
  end
...
    context 'store_attribute_with_nil_value = true', config: { store_attribute_with_nil_value: true } do
      it 'can set gsi field to nil' do
        doc = klass_with_gsi.new
        doc.name = 'abc'
        doc.age = 1
        doc.save!
        doc.age = nil
        doc.save! #      Aws::DynamoDB::Errors::ValidationException: Invalid attribute value type
      end
    end

Comment thread lib/dynamoid/transaction_write/save.rb Outdated
# changed attributes to persist
changes = @model.attributes.slice(*@model.changed.map(&:to_sym))
changes_dumped = Dynamoid::Dumping.dump_attributes(changes, @model_class.attributes)
changes_dumped = sanitize_item(changes_dumped)

andrykonchin Apr 19, 2025
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It seems it's more difficult than just skipping nil values. A nullified attribute of an existing item simply will not be updated at all (so old value will be kept unchanged). So correct logic is to remove such attribute at all.

It's done in the following way in non-transactional method #save:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Indeed. The code I put in there just skips the nils which is not correct. I'll have to do the removing for all these methods like you said.

Copy link
Copy Markdown
Contributor Author

This was bothering me again today. I'll renew my efforts to fix it.

ckhsponge force-pushed the ckh/transaction-sanitize-item branch from 538b01f to 527c7ef Compare August 28, 2025 17:18

Copy link
Copy Markdown
Contributor Author

@andrykonchin A review of this would be appreciated. Let me know if you see some areas for cleanup.

Copy link
Copy Markdown
Member

I'll review this later this week.

Comment thread lib/dynamoid/transaction_write/base.rb Outdated

Copy link
Copy Markdown
Contributor Author

@andrykonchin Please review again.

andrykonchin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Looks good!

andrykonchin merged commit 6fe3bfc into Dynamoid:master Sep 6, 2025
69 checks passed
ckhsponge deleted the ckh/transaction-sanitize-item branch September 7, 2025 00:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug with null in GSI in transaction

2 participants


Back | FazBrowse Home | New Git URL