Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/palkan/isolator
Detect non-atomic interactions within DB transactions
https://github.com/palkan/isolator
activerecord developer-tools hacktoberfest rails testing-tools
Last synced: about 2 months ago
JSON representation
Detect non-atomic interactions within DB transactions
- Host: GitHub
- URL: https://github.com/palkan/isolator
- Owner: palkan
- License: mit
- Created: 2018-01-28T09:57:47.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2024-04-26T18:20:05.000Z (5 months ago)
- Last Synced: 2024-07-18T20:28:45.053Z (2 months ago)
- Topics: activerecord, developer-tools, hacktoberfest, rails, testing-tools
- Language: Ruby
- Homepage:
- Size: 235 KB
- Stars: 848
- Watchers: 9
- Forks: 26
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Funding: .github/FUNDING.yml
- License: LICENSE.txt
Awesome Lists containing this project
README
[](http://cultofmartians.com/tasks/isolator.html)
[](https://badge.fury.io/rb/isolator)
# Isolator
Detect non-atomic interactions within DB transactions.
Examples:
```ruby
# HTTP calls within transaction
User.transaction do
user = User.new(user_params)
user.save!
# HTTP API call
PaymentsService.charge!(user)
end
#=> raises Isolator::HTTPError
# background job
User.transaction do
user.update!(confirmed_at: Time.now)
UserMailer.successful_confirmation(user).deliver_later
end
#=> raises Isolator::BackgroundJobError
```
Of course, Isolator can detect _implicit_ transactions too. Consider this pretty common bad practice–enqueueing background job from `after_create` callback:
```ruby
class Comment < ApplicationRecord
# the good way is to use after_create_commit
# (or not use callbacks at all)
after_create :notify_author
private
def notify_author
CommentMailer.comment_created(self).deliver_later
end
end
Comment.create(text: "Mars is watching you!")
#=> raises Isolator::BackgroundJobError
```
Isolator is supposed to be used in tests and on staging.
## Installation
Add this line to your application's Gemfile:
```ruby
# We suppose that Isolator is used in development and test
# environments.
group :development, :test do
gem "isolator"
end
# Or you can add it to Gemfile with `require: false`
# and require it manually in your code.
#
# This approach is useful when you want to use it in staging env too.
gem "isolator", require: false
```
## Usage
Isolator is a plug-n-play tool, so, it begins to work right after required.
However, there are some potential caveats:
1) Isolator tries to detect the environment automatically and includes only necessary adapters. Thus the order of loading gems matters: make sure that `isolator` is required in the end (NOTE: in Rails, all adapters loaded after application initialization).
2) Isolator does not distinguish framework-level adapters. For example, `:active_job` spy doesn't take into account which AJ adapter you use; if you are using a safe one (e.g. `Que`) just disable the `:active_job` adapter to avoid false negatives. You can do this by adding an initializer:
```rb
require "active_job/base"
Isolator.adapters.active_job.disable!
```
4) Isolator tries to detect the `test` environment and slightly change its behavior: first, it respect _transactional tests_; secondly, error raising is turned on by default (see [below](#configuration)).
5) Experimental [multiple databases](https://guides.rubyonrails.org/active_record_multiple_databases.html) has been added in v0.7.0. Please, let us know if you encounter any issues.
### Configuration
```ruby
Isolator.configure do |config|
# Specify a custom logger to log offenses
config.logger = nil
# Raise exception on offense
config.raise_exceptions = false # true in test env
# Send notifications to uniform_notifier
config.send_notifications = false
# Customize backtrace filtering (provide a callable)
# By default, just takes the top-5 lines
config.backtrace_filter = ->(backtrace) { backtrace.take(5) }
# Define a custom ignorer class (must implement .prepare)
# uses a row number based list from the .isolator_todo.yml file
config.ignorer = Isolator::Ignorer
# Turn on/off raising exceptions for simultaneous transactions to different databases
config.disallow_cross_database_transactions = false
end
```
Isolator relies on [uniform_notifier][] to send custom notifications.
**NOTE:** `uniform_notifier` should be installed separately (i.e., added to Gemfile).
### Callbacks
Isolator different callbacks so you can inject your own logic or build custom extensions.
```ruby
# This callback is called when Isolator enters the "danger zone"—a within-transaction context
Isolator.before_isolate do
puts "Entering a database transaction. Be careful!"
end
# This callback is called when Isolator leaves the "danger zone"
Isolator.after_isolate do
puts "Leaving a database transaction. Everything is fine. Feel free to call slow HTTP APIs"
end
# This callback is called every time a new transaction is open (root or nested)
Isolator.on_transaction_open do |event|
puts "New transaction from #{event[:connection_id]}. " \
"Current depth: #{event[:depth]}"
end
# This callback is called every time a transaction is completed
Isolator.on_transaction_close do |event|
puts "Transaction completed from #{event[:connection_id]}. " \
"Current depth: #{event[:depth]}"
end
```
### Transactional tests support
- Rails' baked-in [use_transactional_tests](https://api.rubyonrails.org/classes/ActiveRecord/FixtureSet.html#class-ActiveRecord::FixtureSet-label-Transactional+Tests)
- [database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) gem. Make sure that you require isolator _after_ database_cleaner.
### Supported ORMs
- `ActiveRecord` >= 6.0 (see older versions of Isolator for previous versions)
- `ROM::SQL` (only if Active Support instrumentation extension is loaded)
### Adapters
Isolator has a bunch of built-in adapters:
- `:http` – built on top of [Sniffer][]
- `:active_job`
- `:sidekiq`
- `:resque`
- `:resque_scheduler`
- `:sucker_punch`
- `:mailer`
- `:webmock` – track mocked HTTP requests (unseen by Sniffer) in tests
- `:action_cable`
You can dynamically enable/disable adapters, e.g.:
```ruby
# Disable HTTP adapter == do not spy on HTTP requests
Isolator.adapters.http.disable!
# Enable back
Isolator.adapters.http.enable!
```
For `active_job`, be sure to first `require "active_job/base"`.
### Fix Offenses
For the actions that should be executed only after successful transaction commit (which is mostly always so), you can try to use the `after_commit` callback from [after_commit_everywhere] gem (or use native AR callback in models if it's applicable).
### Ignore Offenses
Since Isolator adapter is just a wrapper over original code, it may lead to false positives when there is another library patching the same behaviour. In that case you might want to ignore some offenses.
Consider an example: we use Sidekiq along with [`sidekiq-postpone`](https://github.com/marshall-lee/sidekiq-postpone)–gem that patches `Sidekiq::Client#raw_push` and allows you to postpone jobs enqueueing (e.g. to enqueue everything when a transaction is commited–we don't want to raise exceptions in such situation).
To ignore offenses when `sidekiq-postpone` is active, you can add an ignore `proc`:
```ruby
Isolator.adapters.sidekiq.ignore_if { Thread.current[:sidekiq_postpone] }
```
You can add as many _ignores_ as you want, the offense is registered iff all of them return false.
### Using with sidekiq/testing
If you require sidekiq/testing in your tests after isolator is required then it will blow away isolator's hooks, so you need to require isolator after requiring sidekiq/testing.
If you're using Rails and want to use isolator in development and staging, then here is a way to do this.
```ruby
# Gemfile
gem "isolator", require: false # so it delays loading till after sidekiq/testing
# config/initializers/isolator.rb
require "sidekiq/testing" if Rails.env.test?
unless Rails.env.production? # so we get it in staging too
require "isolator"
Isolator.configure do |config|
config.send_notifications = true # ...
end
end
```
### Using with legacy Rails codebases
If you already have a huge Rails project it can be tricky to turn Isolator on because you'll immediately get a lot of failed specs. If you want to fix detected issues one by one, you can list all of them in the special files `.isolator_todo.yml` and `.isolator_ignore.yml` in the following way:
```
sidekiq:
- app/models/user.rb:20
- app/models/sales/**/*.rb
```
You can ignore the same files in multiple adapters using YML aliases in the following way:
```
http_common: &http_common
- app/models/user.rb:20
http: *http_common
webmock: *http_common
```
All the exceptions raised in the listed lines will be ignored.
The `.isolator_todo.yml` file is intended to point to the code that should be fixed later, and `.isolator_ignore.yml` points to the code that for some reasons is not expected to be fixed. (See https://github.com/palkan/isolator/issues/40)
### Using with legacy Ruby codebases
If you are not using Rails, you'll have to load ignores from file manually, using `Isolator::Ignorer.prepare(path:)`, for instance `Isolator::Ignorer.prepare(path: "./config/.isolator_todo.yml")`
## Custom Adapters
An adapter is just a combination of a _method wrapper_ and lifecycle hooks.
Suppose that you have a class `Danger` with a method `#explode`, which is not safe to be run within a DB transaction. Then you can _isolate_ it (i.e., register with Isolator):
```ruby
# The first argument is a unique adapter id,
# you can use it later to enable/disable the adapter
#
# The second argument is the method owner and
# the third one is a method name.
Isolator.isolate :danger, Danger, :explode, options
# NOTE: if you want to isolate a class method, use singleton_class instead
Isolator.isolate :danger, Danger.singleton_class, :explode, options
```
Possible `options` are:
- `exception_class` – an exception class to raise in case of offense
- `exception_message` – custom exception message (could be specified without a class)
- `details_message` – a block to generate additional exception message information:
```ruby
Isolator.isolate :active_job,
target: ActiveJob::Base,
method_name: :enqueue,
exception_class: Isolator::BackgroundJobError,
details_message: ->(obj) {
"#{obj.class.name}(#{obj.arguments})"
}
Isolator.isolate :promoter,
target: UserPromoter,
method_name: :call,
details_message: ->(obj_, args, kwargs) {
# UserPromoter.call(user, role, by: nil)
user, role = args
by = kwargs[:by]
"#{user.name} promoted to #{role} by #{by&.name || "system"})"
}
```
Trying to register the same adapter name twice will raise an error. You can guard for it, or remove old adapters before in order to replace them.
```ruby
unless Isolator.has_adapter?(:promoter)
Isolator.isolate(:promoter, *rest)
end
```
```ruby
# Handle code reloading
class Messager
end
Isolator.remove_adapter(:messager)
Isolator.isolate(:messager, target: Messager, **rest)
```
You can also add some callbacks to be run before and after the transaction:
```ruby
Isolator.before_isolate do
# right after we enter the transaction
end
Isolator.after_isolate do
# right after the transaction has been committed/rolled back
end
```
## Troubleshooting
### Verbose output
In most cases, turning on verbose output for Isolator helps to identify the issue. To do that, you can either specify `ISOLATOR_DEBUG=true` environment variable or set `Isolator.debug_enabled` manually.
### Tests failing after upgrading to Rails 6.0.3 while using [Combustion](https://github.com/pat/combustion)
The reason is that Rails started using a [separate connection pool for advisory locks](https://github.com/rails/rails/pull/38235) since 6.0.3. Since Combustion usually applies migrations for every test run, this pool becomse visible to [test fixtures](https://github.com/rails/rails/blob/b738f1930f3c82f51741ef7241c1fee691d7deb2/activerecord/lib/active_record/test_fixtures.rb#L123-L127), which resulted in 2 transactional commits tracked by Isolator, which only expects one. That leads to false negatives.
To fix this disable migrations advisory locks by adding `advisory_locks: false` to your database configuration in `(spec|test)/internal/config/database.yml`.
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/palkan/isolator.
## License
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
[Sniffer]: https://github.com/aderyabin/sniffer
[uniform_notifier]: https://github.com/flyerhzm/uniform_notifier
[after_commit_everywhere]: https://github.com/Envek/after_commit_everywhere