diff --git a/elasticsearch-persistence/.gitignore b/elasticsearch-persistence/.gitignore new file mode 100644 index 000000000..d87d4be66 --- /dev/null +++ b/elasticsearch-persistence/.gitignore @@ -0,0 +1,17 @@ +*.gem +*.rbc +.bundle +.config +.yardoc +Gemfile.lock +InstalledFiles +_yardoc +coverage +doc/ +lib/bundler/man +pkg +rdoc +spec/reports +test/tmp +test/version_tmp +tmp diff --git a/elasticsearch-persistence/Gemfile b/elasticsearch-persistence/Gemfile new file mode 100644 index 000000000..a60150cf4 --- /dev/null +++ b/elasticsearch-persistence/Gemfile @@ -0,0 +1,4 @@ +source 'https://rubygems.org' + +# Specify your gem's dependencies in elasticsearch-persistence.gemspec +gemspec diff --git a/elasticsearch-persistence/LICENSE.txt b/elasticsearch-persistence/LICENSE.txt new file mode 100644 index 000000000..489007102 --- /dev/null +++ b/elasticsearch-persistence/LICENSE.txt @@ -0,0 +1,13 @@ +Copyright (c) 2014 Elasticsearch + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/elasticsearch-persistence/README.md b/elasticsearch-persistence/README.md new file mode 100644 index 000000000..b02aa99a7 --- /dev/null +++ b/elasticsearch-persistence/README.md @@ -0,0 +1,424 @@ +# Elasticsearch::Persistence + +This library provides a persistence layer for Ruby domain objects in Elasticsearch, +using the Repository and ActiveRecord patterns. + +The library is compatible with Ruby 1.9.3 (or higher) and Elasticsearch 1.0 (or higher). + +## Installation + +Install the package from [Rubygems](https://rubygems.org): + + gem install elasticsearch-persistence + +To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://bundler.io): + + gem 'elasticsearch-persistence', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' + +or install it from a source code checkout: + + git clone https://github.com/elasticsearch/elasticsearch-rails.git + cd elasticsearch-rails/elasticsearch-persistence + bundle install + rake install + +## Usage + +### The Repository Pattern + +The `Elasticsearch::Persistence::Repository` module provides an implementation of the +[repository pattern](http://martinfowler.com/eaaCatalog/repository.html) and allows +to save, delete, find and search objects stored in Elasticsearch, as well as configure +mappings and settings for the index. + +Let's have a simple, plain old Ruby object (PORO): + +```ruby +class Note + attr_reader :attributes + + def initialize(attributes={}) + @attributes = attributes + end + + def to_hash + @attributes + end +end +``` + +Let's create a default, "dumb" repository, as a first step: + +```ruby +require 'elasticsearch/persistence' +repository = Elasticsearch::Persistence::Repository.new +``` + +We can save a `Note` instance into the repository... + +```ruby +note = Note.new id: 1, text: 'Test' + +repository.save(note) +# PUT http://localhost:9200/repository/note/1 [status:201, request:0.210s, query:n/a] +# > {"id":1,"text":"Test"} +# < {"_index":"repository","_type":"note","_id":"1","_version":1,"created":true} +``` + +...find it... + +```ruby +n = repository.find(1) +# GET http://localhost:9200/repository/_all/1 [status:200, request:0.003s, query:n/a] +# < {"_index":"repository","_type":"note","_id":"1","_version":2,"found":true, "_source" : {"id":1,"text":"Test"}} +=> 1, "text"=>"Test"}> +``` + +...search for it... + +```ruby +repository.search(query: { match: { text: 'test' } }).first +# GET http://localhost:9200/repository/_search [status:200, request:0.005s, query:0.002s] +# > {"query":{"match":{"text":"test"}}} +# < {"took":2, ... "hits":{"total":1, ... "hits":[{ ... "_source" : {"id":1,"text":"Test"}}]}} +=> 1, "text"=>"Test"}> +``` + +...or delete it: + +```ruby +repository.delete(note) +# DELETE http://localhost:9200/repository/note/1 [status:200, request:0.014s, query:n/a] +# < {"found":true,"_index":"repository","_type":"note","_id":"1","_version":3} +=> {"found"=>true, "_index"=>"repository", "_type"=>"note", "_id"=>"1", "_version"=>2} +``` + +The repository module provides a number of features and facilities to configure and customize the behaviour: + +* Configuring the Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage) being used +* Setting the index name, document type, and object class for deserialization +* Composing mappings and settings for the index +* Creating, deleting or refreshing the index +* Finding or searching for documents +* Providing access both to domain objects and hits for search results +* Providing access to the Elasticsearch response for search results (aggregations, total, ...) +* Defining the methods for serialization and deserialization + +You can use the default repository class, or include the module in your own. Let's review it in detail. + +#### The Default Class + +For simple cases, you can use the default, bundled repository class, and configure/customize it: + +```ruby +repository = Elasticsearch::Persistence::Repository.new do + # Configure the Elasticsearch client + client Elasticsearch::Client.new url: ENV['ELASTICSEARCH_URL'], log: true + + # Set a custom index name + index :my_notes + + # Set a custom document type + type :my_note + + # Specify the class to inicialize when deserializing documents + klass Note + + # Configure the settings and mappings for the Elasticsearch index + settings number_of_shards: 1 do + mapping do + indexes :text, analyzer: 'snowball' + end + end + + # Customize the serialization logic + def serialize(document) + super.merge(my_special_key: 'my_special_stuff') + end + + # Customize the de-serialization logic + def deserialize(document) + puts "# ***** CUSTOM DESERIALIZE LOGIC KICKING IN... *****" + super + end +end +``` + +The custom Elasticsearch client will be used now, with a custom index and type names, +as well as the custom serialization and de-serialization logic. + +We can create the index with the desired settings and mappings: + +```ruby +repository.create_index! force: true +# PUT http://localhost:9200/my_notes +# > {"settings":{"number_of_shards":1},"mappings":{ ... {"text":{"analyzer":"snowball","type":"string"}}}}} +``` + +Save the document with extra properties added by the `serialize` method: + +```ruby +repository.save(note) +# PUT http://localhost:9200/my_notes/my_note/1 +# > {"id":1,"text":"Test","my_special_key":"my_special_stuff"} +{"_index"=>"my_notes", "_type"=>"my_note", "_id"=>"1", "_version"=>4, ... } +``` + +And `deserialize` it: + +```ruby +repository.find(1) +# ***** CUSTOM DESERIALIZE LOGIC KICKING IN... ***** +"my_special_stuff"}> +``` + +#### A Custom Class + +In most cases, though, you'll want to use a custom class for the repository, so let's do that: + +```ruby +require 'base64' + +class NoteRepository + include Elasticsearch::Persistence::Repository + + def initialize(options={}) + index options[:index] || 'notes' + client Elasticsearch::Client.new url: options[:url], log: options[:log] + end + + klass Note + + settings number_of_shards: 1 do + mapping do + indexes :text, analyzer: 'snowball' + # Do not index images + indexes :image, index: 'no' + end + end + + # Base64 encode the "image" field in the document + # + def serialize(document) + hash = document.to_hash.clone + hash['image'] = Base64.encode64(hash['image']) if hash['image'] + hash.to_hash + end + + # Base64 decode the "image" field in the document + # + def deserialize(document) + hash = document['_source'] + hash['image'] = Base64.decode64(hash['image']) if hash['image'] + klass.new hash + end +end +``` + +Include the `Elasticsearch::Persistence::Repository` module to add the repository methods into the class. + +You can customize the repository in the familiar way, by calling the DSL-like methods. + +You can implement a custom initializer for your repository, add complex logic in its +class and instance methods -- in general, have all the freedom of a standard Ruby class. + +```ruby +repository = NoteRepository.new url: 'http://localhost:9200', log: true + +# Configure the repository instance +repository.index = 'notes_development' +repository.client.transport.logger.formatter = proc { |s, d, p, m| "\e[2m# #{m}\n\e[0m" } + +repository.create_index! force: true + +note = Note.new 'id' => 1, 'text' => 'Document with image', 'image' => '... BINARY DATA ...' + +repository.save(note) +# PUT http://localhost:9200/notes_development/note/1 +# > {"id":1,"text":"Document with image","image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"} +puts repository.find(1).attributes['image'] +# GET http://localhost:9200/notes_development/note/1 +# < {... "_source" : { ... "image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"}} +# => ... BINARY DATA ... +``` + +#### Methods Provided by the Repository + +##### Client + +The repository uses the standard Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage), +which is accessible with the `client` getter and setter methods: + +```ruby +repository.client = Elasticsearch::Client.new url: 'http://search.server.org' +repository.client.transport.logger = Logger.new(STDERR) +``` + +##### Naming + +The `index` method specifies the Elasticsearch index to use for storage, lookup and search +(when not set, the value is inferred from the repository class name): + +```ruby +repository.index = 'notes_development' +``` + +The `type` method specifies the Elasticsearch document type to use for storage, lookup and search +(when not set, the value is inferred from the document class name, or `_all` is used): + +```ruby +repository.type = 'my_note' +``` + +The `klass` method specifies the Ruby class name to use when initializing objects from +documents retrieved from the repository (when not set, the value is inferred from the +document `_type` as fetched from Elasticsearch): + +```ruby +repository.klass = MyNote +``` + +##### Index Configuration + +The `settings` and `mappings` methods, provided by the +[`elasticsearch-model`](http://rubydoc.info/gems/elasticsearch-model/Elasticsearch/Model/Indexing/ClassMethods) +gem, allow to configure the index properties: + +```ruby +repository.settings number_of_shards: 1 +repository.settings.to_hash +# => {:number_of_shards=>1} + +repository.mappings { indexes :title, analyzer: 'snowball' } +repository.mappings.to_hash +# => { :note => {:properties=> ... }} +``` + +The convenience methods `create_index!`, `delete_index!` and `refresh_index!` allow you to manage the index lifecycle. + +##### Serialization + +The `serialize` and `deserialize` methods allow you to customize the serialization of the document when passing it +to the storage, and the initialization procedure when loading it from the storage: + +```ruby +class NoteRepository + def serialize(document) + Hash[document.to_hash.map() { |k,v| v.upcase! if k == :title; [k,v] }] + end + def deserialize(document) + MyNote.new ActiveSupport::HashWithIndifferentAccess.new(document['_source']).deep_symbolize_keys + end +end +``` + +##### Storage + +The `save` method allows you to store a domain object in the repository: + +```ruby +note = Note.new id: 1, title: 'Quick Brown Fox' +repository.save(note) +# => {"_index"=>"notes_development", "_type"=>"my_note", "_id"=>"1", "_version"=>1, "created"=>true} +``` + +The `delete` method allows to remove objects from the repository (pass either the object itself or its ID): + +```ruby +repository.delete(note) +repository.delete(1) +``` + +##### Finding + +The `find` method allows to find one or many documents in the storage and returns them as deserialized Ruby objects: + +```ruby +repository.save Note.new(id: 2, title: 'Fast White Dog') + +note = repository.find(1) +# => + +notes = repository.find(1, 2) +# => [, ] +``` + +When the document with a specific ID isn't found, a `nil` is returned instead of the deserialized object: + +```ruby +notes = repository.find(1, 3, 2) +# => [, nil, ] +``` + +Handle the missing objects in the application code, or call `compact` on the result. + +##### Search + +The `search` method to retrieve objects from the repository by a query string or definition in the Elasticsearch DSL: + +```ruby +repository.search('fox or dog').to_a +# GET http://localhost:9200/notes_development/my_note/_search?q=fox +# => [, ] + +repository.search(query: { match: { title: 'fox dog' } }).to_a +# GET http://localhost:9200/notes_development/my_note/_search +# > {"query":{"match":{"title":"fox dog"}}} +# => [, ] +``` + +The returned object is an instance of the `Elasticsearch::Persistence::Repository::Response::Results` class, +which provides access to the results, the full returned response and hits. + +```ruby +results = repository.search(query: { match: { title: 'fox dog' } }) + +# Iterate over the objects +# +results.each do |note| + puts "* #{note.attributes[:title]}" +end +# * QUICK BROWN FOX +# * FAST WHITE DOG + +# Iterate over the objects and hits +# +results.each_with_hit do |note, hit| + puts "* #{note.attributes[:title]}, score: #{hit._score}" +end +# * QUICK BROWN FOX, score: 0.29930896 +# * FAST WHITE DOG, score: 0.29930896 + +# Get total results +# +results.total +# => 2 + +# Access the raw response as a Hashie::Mash instance +results.response._shards.failed +# => 0 +``` + +### The ActiveRecord Pattern + +_Work in progress_. The ActiveRecord [pattern](http://www.martinfowler.com/eaaCatalog/activeRecord.html) will work +in a very similar way as `Tire::Model::Persistence`, allowing a drop-in replacement of an Elasticsearch-backed model +in Ruby on Rails applications. + +## License + +This software is licensed under the Apache 2 license, quoted below. + + Copyright (c) 2014 Elasticsearch + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/elasticsearch-persistence/Rakefile b/elasticsearch-persistence/Rakefile new file mode 100644 index 000000000..303f43997 --- /dev/null +++ b/elasticsearch-persistence/Rakefile @@ -0,0 +1,57 @@ +require "bundler/gem_tasks" + +desc "Run unit tests" +task :default => 'test:unit' +task :test => 'test:unit' + +# ----- Test tasks ------------------------------------------------------------ + +require 'rake/testtask' +namespace :test do + task :ci_reporter do + ENV['CI_REPORTS'] ||= 'tmp/reports' + require 'ci/reporter/rake/minitest' + Rake::Task['ci:setup:minitest'].invoke + end + + Rake::TestTask.new(:unit) do |test| + Rake::Task['test:ci_reporter'].invoke if ENV['CI'] + test.libs << 'lib' << 'test' + test.test_files = FileList["test/unit/**/*_test.rb"] + # test.verbose = true + # test.warning = true + end + + Rake::TestTask.new(:integration) do |test| + Rake::Task['test:ci_reporter'].invoke if ENV['CI'] + test.libs << 'lib' << 'test' + test.test_files = FileList["test/integration/**/*_test.rb"] + end + + Rake::TestTask.new(:all) do |test| + Rake::Task['test:ci_reporter'].invoke if ENV['CI'] + test.libs << 'lib' << 'test' + test.test_files = FileList["test/unit/**/*_test.rb", "test/integration/**/*_test.rb"] + end +end + +# ----- Documentation tasks --------------------------------------------------- + +require 'yard' +YARD::Rake::YardocTask.new(:doc) do |t| + t.options = %w| --embed-mixins --markup=markdown | +end + +# ----- Code analysis tasks --------------------------------------------------- + +if defined?(RUBY_VERSION) && RUBY_VERSION > '1.9' + begin + require 'cane/rake_task' + Cane::RakeTask.new(:quality) do |cane| + cane.abc_max = 15 + cane.style_measure = 120 + end + rescue LoadError + warn "cane not available, quality task not provided." + end +end diff --git a/elasticsearch-persistence/elasticsearch-persistence.gemspec b/elasticsearch-persistence/elasticsearch-persistence.gemspec new file mode 100644 index 000000000..cb66916ed --- /dev/null +++ b/elasticsearch-persistence/elasticsearch-persistence.gemspec @@ -0,0 +1,46 @@ +# coding: utf-8 +lib = File.expand_path('../lib', __FILE__) +$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) +require 'elasticsearch/persistence/version' + +Gem::Specification.new do |s| + s.name = "elasticsearch-persistence" + s.version = Elasticsearch::Persistence::VERSION + s.authors = ["Karel Minarik"] + s.email = ["karel.minarik@elasticsearch.org"] + s.description = "Persistence layer for Ruby models and Elasticsearch." + s.summary = "Persistence layer for Ruby models and Elasticsearch." + s.homepage = "https://github.com/elasticsearch/elasticsearch-rails/" + s.license = "Apache 2" + + s.files = `git ls-files -z`.split("\x0") + s.executables = s.files.grep(%r{^bin/}) { |f| File.basename(f) } + s.test_files = s.files.grep(%r{^(test|spec|features)/}) + s.require_paths = ["lib"] + + s.extra_rdoc_files = [ "README.md", "LICENSE.txt" ] + s.rdoc_options = [ "--charset=UTF-8" ] + + s.add_dependency "elasticsearch", '> 0.4' + s.add_dependency "elasticsearch-model", '>= 0.1' + s.add_dependency "activesupport", '> 3' + s.add_dependency "hashie" + + s.add_development_dependency "bundler", "~> 1.5" + s.add_development_dependency "rake" + + s.add_development_dependency "elasticsearch-extensions" + + s.add_development_dependency "shoulda-context" + s.add_development_dependency "mocha" + s.add_development_dependency "turn" + s.add_development_dependency "yard" + s.add_development_dependency "ruby-prof" + s.add_development_dependency "pry" + s.add_development_dependency "ci_reporter" + + if defined?(RUBY_VERSION) && RUBY_VERSION > '1.9' + s.add_development_dependency "simplecov" + s.add_development_dependency "cane" + end +end diff --git a/elasticsearch-persistence/examples/sinatra/.gitignore b/elasticsearch-persistence/examples/sinatra/.gitignore new file mode 100644 index 000000000..e9d847d61 --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/.gitignore @@ -0,0 +1,7 @@ +.DS_Store +Gemfile.lock +tmp/* +log/* +doc/ +.yardoc +.vagrant diff --git a/elasticsearch-persistence/examples/sinatra/Gemfile b/elasticsearch-persistence/examples/sinatra/Gemfile new file mode 100644 index 000000000..63d5d75c5 --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/Gemfile @@ -0,0 +1,28 @@ +source 'https://rubygems.org' + +gem 'rake' +gem 'ansi' + +gem 'multi_json' +gem 'oj' +gem 'hashie' + +gem 'patron' +gem 'elasticsearch' +gem 'elasticsearch-model', path: File.expand_path('../../../../elasticsearch-model', __FILE__) +gem 'elasticsearch-persistence', path: File.expand_path('../../../', __FILE__) + +gem 'sinatra', require: false +gem 'thin' + +group :development do + gem 'sinatra-contrib' +end + +group :test do + gem 'elasticsearch-extensions' + gem 'rack-test' + gem 'shoulda-context' + gem 'turn' + gem 'mocha' +end diff --git a/elasticsearch-persistence/examples/sinatra/README.markdown b/elasticsearch-persistence/examples/sinatra/README.markdown new file mode 100644 index 000000000..2aa15ec32 --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/README.markdown @@ -0,0 +1,36 @@ +Demo Aplication for the Repository Pattern +========================================== + +This directory contains a simple demo application for the repository pattern of the `Elasticsearch::Persistence` +module in the [Sinatra](http://www.sinatrarb.com) framework. + +To run the application, first install the required gems and start the application: + +``` +bundle install +bundle exec ruby application.rb +``` + +The application demonstrates: + +* How to use a plain old Ruby object (PORO) as the domain model +* How to set up, configure and use the repository instance +* How to use the repository in tests + +## License + +This software is licensed under the Apache 2 license, quoted below. + + Copyright (c) 2014 Elasticsearch + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/elasticsearch-persistence/examples/sinatra/application.rb b/elasticsearch-persistence/examples/sinatra/application.rb new file mode 100644 index 000000000..50f56dece --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/application.rb @@ -0,0 +1,238 @@ +$LOAD_PATH.unshift File.expand_path('../../../lib/', __FILE__) + +require 'sinatra/base' + +require 'multi_json' +require 'oj' +require 'hashie/mash' + +require 'elasticsearch' +require 'elasticsearch/model' +require 'elasticsearch/persistence' + +class Note + attr_reader :attributes + + def initialize(attributes={}) + @attributes = Hashie::Mash.new(attributes) + __add_date + __extract_tags + __truncate_text + self + end + + def method_missing(method_name, *arguments, &block) + attributes.respond_to?(method_name) ? attributes.__send__(method_name, *arguments, &block) : super + end + + def respond_to?(method_name, include_private=false) + attributes.respond_to?(method_name) || super + end + + def tags; attributes.tags || []; end + + def to_hash + @attributes.to_hash + end + + def __extract_tags + tags = attributes['text'].scan(/(\[\w+\])/).flatten if attributes['text'] + unless tags.nil? || tags.empty? + attributes.update 'tags' => tags.map { |t| t.tr('[]', '') } + attributes['text'].gsub!(/(\[\w+\])/, '').strip! + end + end + + def __add_date + attributes['created_at'] ||= Time.now.utc.iso8601 + end + + def __truncate_text + attributes['text'] = attributes['text'][0...80] + ' (...)' if attributes['text'] && attributes['text'].size > 80 + end +end + +class NoteRepository + include Elasticsearch::Persistence::Repository + + client Elasticsearch::Client.new url: ENV['ELASTICSEARCH_URL'], log: true + + index :notes + type :note + + mapping do + indexes :text, analyzer: 'snowball' + indexes :tags, analyzer: 'keyword' + indexes :created_at, type: 'date' + end + + create_index! + + def deserialize(document) + Note.new document['_source'].merge('id' => document['_id']) + end +end unless defined?(NoteRepository) + +class Application < Sinatra::Base + enable :logging + enable :inline_templates + enable :method_override + + configure :development do + enable :dump_errors + disable :show_exceptions + + require 'sinatra/reloader' + register Sinatra::Reloader + end + + set :repository, NoteRepository.new + set :per_page, 25 + + get '/' do + @page = [ params[:p].to_i, 1 ].max + + @notes = settings.repository.search \ + query: ->(q, t) do + query = if q && !q.empty? + { match: { text: q } } + else + { match_all: {} } + end + + filter = if t && !t.empty? + { term: { tags: t } } + end + + if filter + { filtered: { query: query, filter: filter } } + else + query + end + end.(params[:q], params[:t]), + + sort: [{created_at: {order: 'desc'}}], + + size: settings.per_page, + from: settings.per_page * (@page-1), + + aggregations: { tags: { terms: { field: 'tags' } } }, + + highlight: { fields: { text: { fragment_size: 0, pre_tags: [''],post_tags: [''] } } } + + erb :index + end + + post '/' do + unless params[:text].empty? + @note = Note.new params + settings.repository.save(@note, refresh: true) + end + + redirect back + end + + delete '/:id' do |id| + settings.repository.delete(id, refresh: true) + redirect back + end +end + +Application.run! if $0 == __FILE__ + +__END__ + +@@ layout + + + + Notes + + + + +<%= yield %> + + + +@@ index + +
+

Notes

+
+ +
+
+ +
+

All notes <%= @notes.size %>

+
    + <% @notes.response.aggregations.tags.buckets.each do |term| %> +
  • <%= term['key'] %> <%= term['doc_count'] %>
  • + <% end %> +
+

Add a note

+
+

+

+
+
+ +
+<% if @notes.empty? %> +

No notes found.

+<% end %> + +<% @notes.each_with_hit do |note, hit| %> +
+

+ <%= hit.highlight && hit.highlight.size > 0 ? hit.highlight.text.first : note.text %> + + <% note.tags.each do |tag| %> <%= tag %><% end %> + <%= Time.parse(note.created_at).strftime('%d/%m/%Y %H:%M') %> + +

+

+
+<% end %> + +<% if @notes.size > 0 && @page.next <= @notes.total / settings.per_page %> +

→ Load next

+<% end %> +
diff --git a/elasticsearch-persistence/examples/sinatra/config.ru b/elasticsearch-persistence/examples/sinatra/config.ru new file mode 100644 index 000000000..98f8403ad --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/config.ru @@ -0,0 +1,7 @@ +#\ --port 3000 --server thin + +require File.expand_path('../application', __FILE__) + +map '/' do + run Application +end diff --git a/elasticsearch-persistence/examples/sinatra/test.rb b/elasticsearch-persistence/examples/sinatra/test.rb new file mode 100644 index 000000000..cb9528747 --- /dev/null +++ b/elasticsearch-persistence/examples/sinatra/test.rb @@ -0,0 +1,118 @@ +ENV['RACK_ENV'] = 'test' + +at_exit { Elasticsearch::Test::IntegrationTestCase.__run_at_exit_hooks } if ENV['SERVER'] + +require 'test/unit' +require 'shoulda-context' +require 'mocha/setup' +require 'rack/test' +require 'turn' + +require 'elasticsearch/extensions/test/cluster' +require 'elasticsearch/extensions/test/startup_shutdown' + +require_relative 'application' + +NoteRepository.index_name = 'notes_test' + +class Elasticsearch::Persistence::ExampleApplicationTest < Test::Unit::TestCase + include Rack::Test::Methods + alias :response :last_response + + def app + Application.new + end + + context "Note" do + should "be initialized with a Hash" do + note = Note.new 'foo' => 'bar' + assert_equal 'bar', note.attributes['foo'] + end + + should "add created_at when it's not passed" do + note = Note.new + assert_not_nil note.created_at + assert_match /#{Time.now.year}/, note.created_at + end + + should "not add created_at when it's passed" do + note = Note.new 'created_at' => 'FOO' + assert_equal 'FOO', note.created_at + end + + should "trim long text" do + assert_equal 'Hello World', Note.new('text' => 'Hello World').text + assert_equal 'FOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFOOFO (...)', + Note.new('text' => 'FOO'*200).text + end + + should "delegate methods to attributes" do + note = Note.new 'foo' => 'bar' + assert_equal 'bar', note.foo + end + + should "have tags" do + assert_not_nil Note.new.tags + end + + should "provide a `to_hash` method" do + note = Note.new 'foo' => 'bar' + assert_instance_of Hash, note.to_hash + assert_equal ['created_at', 'foo'], note.to_hash.keys.sort + end + + should "extract tags from the text" do + note = Note.new 'text' => 'Hello [foo] [bar]' + assert_equal 'Hello', note.text + assert_equal ['foo', 'bar'], note.tags + end + end + + context "Application" do + setup do + app.settings.repository.client = Elasticsearch::Client.new \ + hosts: [{ host: 'localhost', port: ENV.fetch('TEST_CLUSTER_PORT', 9250)}], + log: true + app.settings.repository.client.transport.logger.formatter = proc { |s, d, p, m| "\e[2m#{m}\n\e[0m" } + app.settings.repository.create_index! force: true + app.settings.repository.client.cluster.health wait_for_status: 'yellow' + end + + should "have the correct index name" do + assert_equal 'notes_test', app.settings.repository.index + end + + should "display empty page when there are no notes" do + get '/' + assert response.ok?, response.status.to_s + assert_match /No notes found/, response.body.to_s + end + + should "display the notes" do + app.settings.repository.save Note.new('text' => 'Hello') + app.settings.repository.refresh_index! + + get '/' + assert response.ok?, response.status.to_s + assert_match /

\s*Hello/, response.body.to_s + end + + should "create a note" do + post '/', { 'text' => 'Hello World' } + follow_redirect! + + assert response.ok?, response.status.to_s + assert_match /Hello World/, response.body.to_s + end + + should "delete a note" do + app.settings.repository.save Note.new('id' => 'foobar', 'text' => 'Perish...') + delete "/foobar" + follow_redirect! + + assert response.ok?, response.status.to_s + assert_no_match /Perish/, response.body.to_s + end + end + +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence.rb b/elasticsearch-persistence/lib/elasticsearch/persistence.rb new file mode 100644 index 000000000..e6d215ad0 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence.rb @@ -0,0 +1,93 @@ +require 'elasticsearch' +require 'elasticsearch/model/indexing' +require 'hashie' + +require 'active_support/inflector' + +require 'elasticsearch/persistence/version' + +require 'elasticsearch/persistence/client' +require 'elasticsearch/persistence/repository/response/results' +require 'elasticsearch/persistence/repository/naming' +require 'elasticsearch/persistence/repository/serialize' +require 'elasticsearch/persistence/repository/store' +require 'elasticsearch/persistence/repository/find' +require 'elasticsearch/persistence/repository/search' +require 'elasticsearch/persistence/repository/class' +require 'elasticsearch/persistence/repository' + +module Elasticsearch + + # Persistence for Ruby domain objects and models in Elasticsearch + # =============================================================== + # + # `Elasticsearch::Persistence` contains modules for storing and retrieving Ruby domain objects and models + # in Elasticsearch. + # + # == Repository + # + # The repository patterns allows to store and retrieve Ruby objects in Elasticsearch. + # + # require 'elasticsearch/persistence' + # + # class Note + # def to_hash; {foo: 'bar'}; end + # end + # + # repository = Elasticsearch::Persistence::Repository.new + # + # repository.save Note.new + # # => {"_index"=>"repository", "_type"=>"note", "_id"=>"mY108X9mSHajxIy2rzH2CA", ...} + # + # Customize your repository by including the main module in a Ruby class + # class MyRepository + # include Elasticsearch::Persistence::Repository + # + # index 'my_notes' + # klass Note + # + # client Elasticsearch::Client.new log: true + # end + # + # repository = MyRepository.new + # + # repository.save Note.new + # # 2014-04-04 22:15:25 +0200: POST http://localhost:9200/my_notes/note [status:201, request:0.009s, query:n/a] + # # 2014-04-04 22:15:25 +0200: > {"foo":"bar"} + # # 2014-04-04 22:15:25 +0200: < {"_index":"my_notes","_type":"note","_id":"-d28yXLFSlusnTxb13WIZQ", ...} + # + module Persistence + + # :nodoc: + module ClassMethods + + # Get or set the default client for all repositories and models + # + # @example Set and configure the default client + # + # Elasticsearch::Persistence.client Elasticsearch::Client.new host: 'http://localhost:9200', tracer: true + # + # @example Perform an API request through the client + # + # Elasticsearch::Persistence.client.cluster.health + # # => { "cluster_name" => "elasticsearch" ... } + # + def client client=nil + @client = client || @client || Elasticsearch::Client.new + end + + # Set the default client for all repositories and models + # + # @example Set and configure the default client + # + # Elasticsearch::Persistence.client = Elasticsearch::Client.new host: 'http://localhost:9200', tracer: true + # => # + # + def client=(client) + @client = client + end + end + + extend ClassMethods + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/client.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/client.rb new file mode 100644 index 000000000..ece71b12b --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/client.rb @@ -0,0 +1,48 @@ +module Elasticsearch + module Persistence + module Repository + + module Client + + # Get or set the default client for this repository + # + # @example Set and configure the client for the repository class + # + # class MyRepository + # include Elasticsearch::Persistence::Repository + # client Elasticsearch::Client.new host: 'http://localhost:9200', log: true + # end + # + # @example Set and configure the client for this repository instance + # + # repository.client Elasticsearch::Client.new host: 'http://localhost:9200', tracer: true + # + # @example Perform an API request through the client + # + # MyRepository.client.cluster.health + # repository.client.cluster.health + # # => { "cluster_name" => "elasticsearch" ... } + # + def client client=nil + @client = client || @client || Elasticsearch::Persistence.client + end + + # Set the default client for this repository + # + # @example Set and configure the client for the repository class + # + # MyRepository.client = Elasticsearch::Client.new host: 'http://localhost:9200', log: true + # + # @example Set and configure the client for this repository instance + # + # repository.client = Elasticsearch::Client.new host: 'http://localhost:9200', tracer: true + # + def client=(client) + @client = client + @client + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository.rb new file mode 100644 index 000000000..8f08d3fd5 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository.rb @@ -0,0 +1,51 @@ +module Elasticsearch + module Persistence + module GatewayDelegation + def method_missing(method_name, *arguments, &block) + gateway.respond_to?(method_name) ? gateway.__send__(method_name, *arguments, &block) : super + end + + def respond_to?(method_name, include_private=false) + gateway.respond_to?(method_name) || super + end + + def respond_to_missing?(method_name, *) + gateway.respond_to?(method_name) || super + end + end + + module Repository + def self.included(base) + gateway = Elasticsearch::Persistence::Repository::Class.new host: base + + base.class_eval do + define_method :gateway do + @gateway ||= gateway + end + + include GatewayDelegation + end + + (class << base; self; end).class_eval do + define_method :gateway do |&block| + @gateway ||= gateway + @gateway.instance_eval(&block) if block + @gateway + end + + include GatewayDelegation + end + + def base.method_added(name) + if :gateway != name && respond_to?(:gateway) && (gateway.public_methods - Object.public_methods).include?(name) + gateway.define_singleton_method(name, self.new.method(name).to_proc) + end + end + end + + def new(options={}, &block) + Elasticsearch::Persistence::Repository::Class.new( {index: 'repository'}.merge(options), &block ) + end; module_function :new + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/class.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/class.rb new file mode 100644 index 000000000..55b6bc8d4 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/class.rb @@ -0,0 +1,69 @@ +module Elasticsearch + module Persistence + module Repository + + # The default repository class, to be used either directly, or as a gateway in a custom repository class + # + # @example Standalone use + # + # repository = Elasticsearch::Persistence::Repository::Class.new + # # => # + # # > repository.save(my_object) + # # => {"_index"=> ... } + # + # + # @example Shortcut use + # + # repository = Elasticsearch::Persistence::Repository.new + # # => # + # + # @example Configuration via a block + # + # repository = Elasticsearch::Persistence::Repository.new do + # index 'my_notes' + # end + # # => # + # # > repository.save(my_object) + # # => {"_index"=> ... } + # + # @example Accessing the gateway in a custom class + # + # class MyRepository + # include Elasticsearch::Persistence::Repository + # end + # + # repository = MyRepository.new + # + # repository.gateway.client.info + # => {"status"=>200, "name"=>"Venom", ... } + # + class Class + include Elasticsearch::Persistence::Repository::Client + include Elasticsearch::Persistence::Repository::Naming + include Elasticsearch::Persistence::Repository::Serialize + include Elasticsearch::Persistence::Repository::Store + include Elasticsearch::Persistence::Repository::Find + include Elasticsearch::Persistence::Repository::Search + + include Elasticsearch::Model::Indexing::ClassMethods + + attr_reader :options + + def initialize(options={}, &block) + @options = options + index_name options.delete(:index) + block.arity < 1 ? instance_eval(&block) : block.call(self) if block_given? + end + + # Return the "host" class, if this repository is a gateway hosted in another class + # + # @return [nil, Class] + # + def host + options[:host] + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/find.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/find.rb new file mode 100644 index 000000000..c6a9a6a4e --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/find.rb @@ -0,0 +1,73 @@ +module Elasticsearch + module Persistence + module Repository + class DocumentNotFound < StandardError; end + + # Retrieves one or more domain objects from the repository + # + module Find + + # Retrieve a single object or multiple objects from Elasticsearch by ID or IDs + # + # @example Retrieve a single object by ID + # + # repository.find(1) + # # => + # + # @example Retrieve multiple objects by IDs + # + # repository.find(1, 2) + # # => [, + # + # @return [Object,Array] + # + def find(*args) + options = args.last.is_a?(Hash) ? args.pop : {} + ids = args + + if args.size == 1 + id = args.pop + id.is_a?(Array) ? __find_many(id, options) : __find_one(id, options) + else + __find_many args, options + end + end + + # Return if object exists in the repository + # + # @example + # + # repository.exists?(1) + # => true + # + # @return [true, false] + # + def exists?(id, options={}) + type = document_type || (klass ? __get_type_from_class(klass) : '_all') + client.exists( { index: index_name, type: type, id: id }.merge(options) ) + end + + # @api private + # + def __find_one(id, options={}) + type = document_type || (klass ? __get_type_from_class(klass) : '_all') + document = client.get( { index: index_name, type: type, id: id }.merge(options) ) + + deserialize(document) + rescue Elasticsearch::Transport::Transport::Errors::NotFound => e + raise DocumentNotFound, e.message, caller + end + + # @api private + # + def __find_many(ids, options={}) + type = document_type || (klass ? __get_type_from_class(klass) : '_all') + documents = client.mget( { index: index_name, type: type, body: { ids: ids } }.merge(options) ) + + documents['docs'].map { |document| document['found'] ? deserialize(document) : nil } + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/naming.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/naming.rb new file mode 100644 index 000000000..220abbed4 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/naming.rb @@ -0,0 +1,91 @@ +module Elasticsearch + module Persistence + module Repository + + module Naming + + # Get or set the class used to initialize domain objects when deserializing them + # + def klass name=nil + @klass = name || @klass + end + + # Set the class used to initialize domain objects when deserializing them + # + def klass=klass + @klass = klass + end + + # Get or set the index name used when storing and retrieving documents + # + def index_name name=nil + @index_name = name || @index_name || begin + if respond_to?(:host) && host && host.is_a?(Module) + self.host.to_s.underscore.gsub(/\//, '-') + else + self.class.to_s.underscore.gsub(/\//, '-') + end + end + end; alias :index :index_name + + # Set the index name used when storing and retrieving documents + # + def index_name=(name) + @index_name = name + end; alias :index= :index_name= + + # Get or set the document type used when storing and retrieving documents + # + def document_type name=nil + @document_type = name || @document_type || (klass ? klass.to_s.underscore : nil) + end; alias :type :document_type + + # Set the document type used when storing and retrieving documents + # + def document_type=(name) + @document_type = name + end; alias :type= :document_type= + + # Get the Ruby class from the Elasticsearch `_type` + # + # @example + # repository.__get_klass_from_type 'note' + # => Note + # + # @api private + # + def __get_klass_from_type(type) + klass = type.classify + klass.constantize + rescue NameError => e + raise NameError, "Attempted to get class '#{klass}' from the '#{type}' type, but no such class can be found." + end + + # Get the Elasticsearch `_type` from the Ruby class + # + # @example + # repository.__get_type_from_class Note + # => "note" + # + # @api private + # + def __get_type_from_class(klass) + klass.to_s.underscore + end + + # Get a document ID from the document (assuming Hash or Hash-like object) + # + # @example + # repository.__get_id_from_document title: 'Test', id: 'abc123' + # => "abc123" + # + # @api private + # + def __get_id_from_document(document) + document[:id] || document['id'] || document[:_id] || document['_id'] + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/response/results.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/response/results.rb new file mode 100644 index 000000000..fe64ac9b0 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/response/results.rb @@ -0,0 +1,90 @@ +module Elasticsearch + module Persistence + module Repository + module Response # :nodoc: + + # Encapsulates the domain objects and documents returned from Elasticsearch when searching + # + # Implements `Enumerable` and forwards its methods to the {#results} object. + # + class Results + include Enumerable + + attr_reader :repository + + # @param repository [Elasticsearch::Persistence::Repository::Class] The repository instance + # @param response [Hash] The full response returned from the Elasticsearch client + # @param options [Hash] Optional parameters + # + def initialize(repository, response, options={}) + @repository = repository + @response = Hashie::Mash.new(response) + @options = options + end + + def method_missing(method_name, *arguments, &block) + results.respond_to?(method_name) ? results.__send__(method_name, *arguments, &block) : super + end + + def respond_to?(method_name, include_private = false) + results.respond_to?(method_name) || super + end + + # The number of total hits for a query + # + def total + response['hits']['total'] + end + + # The maximum score for a query + # + def max_score + response['hits']['max_score'] + end + + # Yields [object, hit] pairs to the block + # + def each_with_hit(&block) + results.zip(response['hits']['hits']).each(&block) + end + + # Yields [object, hit] pairs and returns the result + # + def map_with_hit(&block) + results.zip(response['hits']['hits']).map(&block) + end + + # Return the collection of domain objects + # + # @example Iterate over the results + # + # results.map { |r| r.attributes[:title] } + # => ["Fox", "Dog"] + # + # @return [Array] + # + def results + @results ||= response['hits']['hits'].map do |document| + repository.deserialize(document.to_hash) + end + end + + # Access the response returned from Elasticsearch by the client + # + # @example Access the aggregations in the response + # + # results = repository.search query: { match: { title: 'fox dog' } }, + # aggregations: { titles: { terms: { field: 'title' } } } + # results.response.aggregations.titles.buckets.map { |term| "#{term['key']}: #{term['doc_count']}" } + # # => ["brown: 1", "dog: 1", ...] + # + # @return [Hashie::Mash] + # + def response + @response + end + end + end + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/search.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/search.rb new file mode 100644 index 000000000..a3843df51 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/search.rb @@ -0,0 +1,59 @@ +module Elasticsearch + module Persistence + module Repository + + # Returns a collection of domain objects by an Elasticsearch query + # + module Search + + # Returns a collection of domain objects by an Elasticsearch query + # + # Pass the query either as a string or a Hash-like object + # + # @example Return objects matching a simple query + # + # repository.search('fox or dog') + # + # @example Return objects matching a query in the Elasticsearch DSL + # + # repository.search(query: { match: { title: 'fox dog' } }) + # + # @example Define additional search parameters, such as highlighted excerpts + # + # results = repository.search(query: { match: { title: 'fox dog' } }, highlight: { fields: { title: {} } }) + # results.map_with_hit { |d,h| h.highlight.title.join } + # # => ["quick brown fox", "fast white dog"] + # + # @example Perform aggregations as part of the request + # + # results = repository.search query: { match: { title: 'fox dog' } }, + # aggregations: { titles: { terms: { field: 'title' } } } + # results.response.aggregations.titles.buckets.map { |term| "#{term['key']}: #{term['doc_count']}" } + # # => ["brown: 1", "dog: 1", ... ] + # + # @example Pass additional options to the search request, such as `size` + # + # repository.search query: { match: { title: 'fox dog' } }, size: 25 + # # GET http://localhost:9200/notes/note/_search + # # > {"query":{"match":{"title":"fox dog"}},"size":25} + # + # @return [Elasticsearch::Persistence::Repository::Response::Results] + # + def search(query_or_definition, options={}) + type = document_type || (klass ? __get_type_from_class(klass) : nil ) + case + when query_or_definition.respond_to?(:to_hash) + response = client.search( { index: index_name, type: type, body: query_or_definition.to_hash }.merge(options) ) + when query_or_definition.is_a?(String) + response = client.search( { index: index_name, type: type, q: query_or_definition }.merge(options) ) + else + raise ArgumentError, "[!] Pass the search definition as a Hash-like object or pass the query as a String" + + " -- #{query_or_definition.class} given." + end + Response::Results.new(self, response) + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/serialize.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/serialize.rb new file mode 100644 index 000000000..027f000b6 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/serialize.rb @@ -0,0 +1,31 @@ +module Elasticsearch + module Persistence + module Repository + + # Provide serialization and deserialization between Ruby objects and Elasticsearch documents + # + # Override these methods in your repository class to customize the logic. + # + module Serialize + + # Serialize the object for storing it in Elasticsearch + # + # In the default implementation, call the `to_hash` method on the passed object. + # + def serialize(document) + document.to_hash + end + + # Deserialize the document retrieved from Elasticsearch into a Ruby object + # + # Use the `klass` property, if defined, otherwise try to get the class from the document's `_type`. + # + def deserialize(document) + _klass = klass || __get_klass_from_type(document['_type']) + _klass.new document['_source'] + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/repository/store.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/store.rb new file mode 100644 index 000000000..07bdce569 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/repository/store.rb @@ -0,0 +1,35 @@ +module Elasticsearch + module Persistence + module Repository + + # Save and delete documents in Elasticsearch + # + module Store + + # Store the serialized object in Elasticsearch + # + def save(document, options={}) + serialized = serialize(document) + id = __get_id_from_document(serialized) + type = document_type || __get_type_from_class(klass || document.class) + client.index( { index: index_name, type: type, id: id, body: serialized }.merge(options) ) + end + + # Remove the serialized object or document with specified ID from Elasticsearch + # + def delete(document, options={}) + if document.is_a?(String) || document.is_a?(Integer) + id = document + type = document_type || __get_type_from_class(klass) + else + serialized = serialize(document) + id = __get_id_from_document(serialized) + type = document_type || __get_type_from_class(klass || document.class) + end + client.delete( { index: index_name, type: type, id: id }.merge(options) ) + end + end + + end + end +end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb new file mode 100644 index 000000000..cf1e84452 --- /dev/null +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb @@ -0,0 +1,5 @@ +module Elasticsearch + module Persistence + VERSION = "0.0.1" + end +end diff --git a/elasticsearch-persistence/test/test_helper.rb b/elasticsearch-persistence/test/test_helper.rb new file mode 100644 index 000000000..5a76e67ae --- /dev/null +++ b/elasticsearch-persistence/test/test_helper.rb @@ -0,0 +1,42 @@ +RUBY_1_8 = defined?(RUBY_VERSION) && RUBY_VERSION < '1.9' + +exit(0) if RUBY_1_8 + +require 'simplecov' and SimpleCov.start { add_filter "/test|test_/" } if ENV["COVERAGE"] + +# Register `at_exit` handler for integration tests shutdown. +# MUST be called before requiring `test/unit`. +at_exit { Elasticsearch::Test::IntegrationTestCase.__run_at_exit_hooks } if ENV['SERVER'] + +require 'test/unit' +require 'shoulda-context' +require 'mocha/setup' +require 'turn' unless ENV["TM_FILEPATH"] || ENV["NOTURN"] || RUBY_1_8 + +require 'ansi' +require 'oj' + +require 'elasticsearch/extensions/test/cluster' +require 'elasticsearch/extensions/test/startup_shutdown' + +require 'elasticsearch/persistence' + +module Elasticsearch + module Test + class IntegrationTestCase < ::Test::Unit::TestCase + extend Elasticsearch::Extensions::Test::StartupShutdown + + startup { Elasticsearch::Extensions::Test::Cluster.start(nodes: 1) if ENV['SERVER'] and not Elasticsearch::Extensions::Test::Cluster.running? } + shutdown { Elasticsearch::Extensions::Test::Cluster.stop if ENV['SERVER'] && started? } + context "IntegrationTest" do; should "noop on Ruby 1.8" do; end; end if RUBY_1_8 + + def setup + tracer = ::Logger.new(STDERR) + tracer.formatter = lambda { |s, d, p, m| "#{m.gsub(/^.*$/) { |n| ' ' + n }.ansi(:faint)}\n" } + Elasticsearch::Persistence.client = Elasticsearch::Client.new \ + host: "localhost:#{(ENV['TEST_CLUSTER_PORT'] || 9250)}", + tracer: (ENV['QUIET'] ? nil : tracer) + end + end + end +end diff --git a/elasticsearch-persistence/test/unit/persistence_test.rb b/elasticsearch-persistence/test/unit/persistence_test.rb new file mode 100644 index 000000000..cd17ba7dc --- /dev/null +++ b/elasticsearch-persistence/test/unit/persistence_test.rb @@ -0,0 +1,32 @@ +require 'test_helper' + +class Elasticsearch::Persistence::ModuleTest < Test::Unit::TestCase + context "The Persistence module" do + + context "client" do + should "have a default client" do + client = Elasticsearch::Persistence.client + assert_not_nil client + assert_instance_of Elasticsearch::Transport::Client, client + end + + should "allow to set a client" do + begin + Elasticsearch::Persistence.client = "Foobar" + assert_equal "Foobar", Elasticsearch::Persistence.client + ensure + Elasticsearch::Persistence.client = nil + end + end + + should "allow to set a client with DSL" do + begin + Elasticsearch::Persistence.client "Foobar" + assert_equal "Foobar", Elasticsearch::Persistence.client + ensure + Elasticsearch::Persistence.client = nil + end + end + end + end +end diff --git a/elasticsearch-persistence/test/unit/repository_class_test.rb b/elasticsearch-persistence/test/unit/repository_class_test.rb new file mode 100644 index 000000000..d29711248 --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_class_test.rb @@ -0,0 +1,51 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryClassTest < Test::Unit::TestCase + context "The default repository class" do + + context "when initialized" do + should "be created from the module" do + repository = Elasticsearch::Persistence::Repository.new + assert_instance_of Elasticsearch::Persistence::Repository::Class, repository + end + + should "store and access the options" do + repository = Elasticsearch::Persistence::Repository::Class.new foo: 'bar' + assert_equal 'bar', repository.options[:foo] + end + + should "instance eval a passed block" do + $foo = 100 + repository = Elasticsearch::Persistence::Repository::Class.new() { $foo += 1 } + assert_equal 101, $foo + end + + should "call a passed block with self" do + foo = 100 + repository = Elasticsearch::Persistence::Repository::Class.new do |r| + assert_instance_of Elasticsearch::Persistence::Repository::Class, r + foo += 1 + end + assert_equal 101, foo + end + + should "configure the index name based on options" do + repository = Elasticsearch::Persistence::Repository::Class.new index: 'foobar' + assert_equal 'foobar', repository.index_name + end + end + + should "include the repository methods" do + repository = Elasticsearch::Persistence::Repository::Class.new + + %w( index_name document_type klass + mappings settings client client= + create_index! delete_index! refresh_index! + save delete serialize deserialize + exists? find search ).each do |method| + assert_respond_to repository, method + end + end + + end +end diff --git a/elasticsearch-persistence/test/unit/repository_client_test.rb b/elasticsearch-persistence/test/unit/repository_client_test.rb new file mode 100644 index 000000000..88e40193e --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_client_test.rb @@ -0,0 +1,32 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryClientTest < Test::Unit::TestCase + context "The repository client" do + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Client }.new + end + + should "have a default client" do + assert_not_nil subject.client + assert_instance_of Elasticsearch::Transport::Client, subject.client + end + + should "allow to set a client" do + begin + subject.client = "Foobar" + assert_equal "Foobar", subject.client + ensure + subject.client = nil + end + end + + should "allow to set the client with DSL" do + begin + subject.client "Foobar" + assert_equal "Foobar", subject.client + ensure + subject.client = nil + end + end + end +end diff --git a/elasticsearch-persistence/test/unit/repository_find_test.rb b/elasticsearch-persistence/test/unit/repository_find_test.rb new file mode 100644 index 000000000..fda7e7105 --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_find_test.rb @@ -0,0 +1,375 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryFindTest < Test::Unit::TestCase + class MyDocument; end + + context "The repository" do + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Find }.new + + @client = mock + @shoulda_subject.stubs(:document_type).returns(nil) + @shoulda_subject.stubs(:klass).returns(nil) + @shoulda_subject.stubs(:index_name).returns('my_index') + @shoulda_subject.stubs(:client).returns(@client) + end + + context "find method" do + should "find one document when passed a single, literal ID" do + subject.expects(:__find_one).with(1, {}) + subject.find(1) + end + + should "find multiple documents when passed multiple IDs" do + subject.expects(:__find_many).with([1, 2], {}) + subject.find(1, 2) + end + + should "find multiple documents when passed an array of IDs" do + subject.expects(:__find_many).with([1, 2], {}) + subject.find([1, 2]) + end + + should "pass the options" do + subject.expects(:__find_one).with(1, { foo: 'bar' }) + subject.find(1, foo: 'bar') + + subject.expects(:__find_many).with([1, 2], { foo: 'bar' }) + subject.find([1, 2], foo: 'bar') + + subject.expects(:__find_many).with([1, 2], { foo: 'bar' }) + subject.find(1, 2, foo: 'bar') + end + end + + context "'exists?' method" do + should "return false when the document does not exist" do + @client.expects(:exists).returns(false) + assert_equal false, subject.exists?('1') + end + + should "return whether document for klass exists" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(MyDocument).at_least_once + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + + @client + .expects(:exists) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns(true) + + assert_equal true, subject.exists?('1') + end + + should "return whether document for document_type exists" do + subject.expects(:document_type).returns('my_document') + subject.expects(:klass).returns(MyDocument).at_most_once + subject.expects(:__get_type_from_class).never + + @client + .expects(:exists) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns(true) + + assert_equal true, subject.exists?('1') + end + + should "return whether document exists" do + subject.expects(:klass).returns(nil) + subject.expects(:__get_type_from_class).never + + @client + .expects(:exists) + .with do |arguments| + assert_equal '_all', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns(true) + + assert_equal true, subject.exists?('1') + end + + should "pass options to the client" do + @client.expects(:exists).with do |arguments| + assert_equal 'foobarbam', arguments[:index] + assert_equal 'bambam', arguments[:routing] + end + + subject.exists? '1', index: 'foobarbam', routing: 'bambam' + end + end + + context "'__find_one' method" do + should "find document based on klass and return a deserialized object" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(MyDocument).at_least_once + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + + subject.expects(:deserialize).with({'_source' => {'foo' => 'bar'}}).returns(MyDocument.new) + + @client + .expects(:get) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns({'_source' => { 'foo' => 'bar' }}) + + assert_instance_of MyDocument, subject.__find_one('1') + end + + should "find document based on document_type and return a deserialized object" do + subject.expects(:document_type).returns('my_document') + subject.expects(:klass).returns(MyDocument).at_most_once + subject.expects(:__get_type_from_class).never + + subject.expects(:deserialize).with({'_source' => {'foo' => 'bar'}}).returns(MyDocument.new) + + @client + .expects(:get) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns({'_source' => { 'foo' => 'bar' }}) + + assert_instance_of MyDocument, subject.__find_one('1') + end + + should "find document and return a deserialized object" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + subject.expects(:deserialize).with({'_source' => {'foo' => 'bar'}}).returns(MyDocument.new) + + @client + .expects(:get) + .with do |arguments| + assert_equal '_all', arguments[:type] + assert_equal '1', arguments[:id] + end + .returns({'_source' => { 'foo' => 'bar' }}) + + assert_instance_of MyDocument, subject.__find_one('1') + end + + should "raise DocumentNotFound exception when the document cannot be found" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(nil).at_least_once + + subject.expects(:deserialize).never + + @client + .expects(:get) + .raises(Elasticsearch::Transport::Transport::Errors::NotFound) + + assert_raise Elasticsearch::Persistence::Repository::DocumentNotFound do + subject.__find_one('foobar') + end + end + + should "pass other exceptions" do + subject.expects(:klass).returns(nil).at_least_once + + subject.expects(:deserialize).never + + @client + .expects(:get) + .raises(RuntimeError) + + assert_raise RuntimeError do + subject.__find_one('foobar') + end + end + + should "pass options to the client" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:deserialize) + + @client + .expects(:get) + .with do |arguments| + assert_equal 'foobarbam', arguments[:index] + assert_equal 'bambam', arguments[:routing] + end + .returns({'_source' => { 'foo' => 'bar' }}) + + subject.__find_one '1', index: 'foobarbam', routing: 'bambam' + end + end + + context "'__find_many' method" do + setup do + @response = {"docs"=> + [ {"_index"=>"my_index", + "_type"=>"note", + "_id"=>"1", + "_version"=>1, + "found"=>true, + "_source"=>{"id"=>"1", "title"=>"Test 1"}}, + + {"_index"=>"my_index", + "_type"=>"note", + "_id"=>"2", + "_version"=>1, + "found"=>true, + "_source"=>{"id"=>"2", "title"=>"Test 2"}} + ]} + end + + should "find documents based on klass and return an Array of deserialized objects" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(MyDocument).at_least_once + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + + subject + .expects(:deserialize) + .with(@response['docs'][0]) + .returns(MyDocument.new) + + subject + .expects(:deserialize) + .with(@response['docs'][1]) + .returns(MyDocument.new) + + @client + .expects(:mget) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal ['1', '2'], arguments[:body][:ids] + end + .returns(@response) + + results = subject.__find_many(['1', '2']) + assert_instance_of MyDocument, results[0] + assert_instance_of MyDocument, results[1] + end + + should "find documents based on document_type and return an Array of deserialized objects" do + subject.expects(:document_type).returns('my_document') + subject.expects(:klass).returns(MyDocument).at_most_once + subject.expects(:__get_type_from_class).never + + subject.expects(:deserialize).twice + + @client + .expects(:mget) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal ['1', '2'], arguments[:body][:ids] + end + .returns(@response) + + subject.__find_many(['1', '2']) + end + + should "find documents and return an Array of deserialized objects" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + subject + .expects(:deserialize) + .with(@response['docs'][0]) + .returns(MyDocument.new) + + subject + .expects(:deserialize) + .with(@response['docs'][1]) + .returns(MyDocument.new) + + @client + .expects(:mget) + .with do |arguments| + assert_equal '_all', arguments[:type] + assert_equal ['1', '2'], arguments[:body][:ids] + end + .returns(@response) + + results = subject.__find_many(['1', '2']) + + assert_equal 2, results.size + + assert_instance_of MyDocument, results[0] + assert_instance_of MyDocument, results[1] + end + + should "find keep missing documents in the result as nil" do + @response = {"docs"=> + [ {"_index"=>"my_index", + "_type"=>"note", + "_id"=>"1", + "_version"=>1, + "found"=>true, + "_source"=>{"id"=>"1", "title"=>"Test 1"}}, + + {"_index"=>"my_index", + "_type"=>"note", + "_id"=>"3", + "_version"=>1, + "found"=>false}, + + {"_index"=>"my_index", + "_type"=>"note", + "_id"=>"2", + "_version"=>1, + "found"=>true, + "_source"=>{"id"=>"2", "title"=>"Test 2"}} + ]} + + subject.expects(:document_type).returns(nil) + subject.expects(:klass).returns(MyDocument).at_least_once + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + + subject + .expects(:deserialize) + .with(@response['docs'][0]) + .returns(MyDocument.new) + + subject + .expects(:deserialize) + .with(@response['docs'][2]) + .returns(MyDocument.new) + + @client + .expects(:mget) + .with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal ['1', '3', '2'], arguments[:body][:ids] + end + .returns(@response) + + results = subject.__find_many(['1', '3', '2']) + + assert_equal 3, results.size + + assert_instance_of MyDocument, results[0] + assert_instance_of NilClass, results[1] + assert_instance_of MyDocument, results[2] + end + + should "pass options to the client" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:deserialize).twice + + @client + .expects(:mget) + .with do |arguments| + assert_equal 'foobarbam', arguments[:index] + assert_equal 'bambam', arguments[:routing] + end + .returns(@response) + + subject.__find_many ['1', '2'], index: 'foobarbam', routing: 'bambam' + end + end + + end +end diff --git a/elasticsearch-persistence/test/unit/repository_indexing_test.rb b/elasticsearch-persistence/test/unit/repository_indexing_test.rb new file mode 100644 index 000000000..5ee22578d --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_indexing_test.rb @@ -0,0 +1,37 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryIndexingTest < Test::Unit::TestCase + context "The repository index methods" do + class MyDocument; end + + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Model::Indexing::ClassMethods }.new + @shoulda_subject.stubs(:index_name).returns('my_index') + @shoulda_subject.stubs(:document_type).returns('my_document') + end + + should "have the convenience index management methods" do + %w( create_index! delete_index! refresh_index! ).each do |method| + assert_respond_to subject, method + end + end + + context "mappings" do + should "configure the mappings for the type" do + subject.mappings do + indexes :title + end + + assert_equal( {:"my_document"=>{:properties=>{:title=>{:type=>"string"}}}}, subject.mappings.to_hash ) + end + end + + context "settings" do + should "configure the settings for the index" do + subject.settings foo: 'bar' + assert_equal( {foo: 'bar'}, subject.settings.to_hash) + end + end + + end +end diff --git a/elasticsearch-persistence/test/unit/repository_module_test.rb b/elasticsearch-persistence/test/unit/repository_module_test.rb new file mode 100644 index 000000000..d85fcf0a7 --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_module_test.rb @@ -0,0 +1,144 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryModuleTest < Test::Unit::TestCase + context "The repository module" do + + class DummyModel + def initialize(attributes={}) + @attributes = attributes + end + + def to_hash + @attributes + end + + def inspect + "" + end + end + + setup do + class DummyRepository + include Elasticsearch::Persistence::Repository + end + end + + teardown do + Elasticsearch::Persistence::RepositoryModuleTest.__send__ :remove_const, :DummyRepository + end + + context "when included" do + should "set up the gateway for the class and instance" do + assert_respond_to DummyRepository, :gateway + assert_respond_to DummyRepository.new, :gateway + + assert_instance_of Elasticsearch::Persistence::Repository::Class, DummyRepository.gateway + assert_instance_of Elasticsearch::Persistence::Repository::Class, DummyRepository.new.gateway + end + + should "proxy repository methods from the class to the gateway" do + class DummyRepository + include Elasticsearch::Persistence::Repository + + index :foobar + klass DummyModel + type :my_dummy_model + mapping { indexes :title, analyzer: 'snowball' } + end + + repository = DummyRepository.new + + assert_equal :foobar, DummyRepository.index + assert_equal DummyModel, DummyRepository.klass + assert_equal :my_dummy_model, DummyRepository.type + assert_equal 'snowball', DummyRepository.mappings.to_hash[:my_dummy_model][:properties][:title][:analyzer] + + assert_equal :foobar, repository.index + assert_equal DummyModel, repository.klass + assert_equal :my_dummy_model, repository.type + assert_equal 'snowball', repository.mappings.to_hash[:my_dummy_model][:properties][:title][:analyzer] + end + + should "correctly delegate to the gateway" do + repository = DummyRepository.new + assert_instance_of Method, repository.method(:index) + end + + should "proxy repository methods from the instance to the gateway" do + class DummyRepository + include Elasticsearch::Persistence::Repository + end + + repository = DummyRepository.new + repository.index :foobar + repository.klass DummyModel + repository.type :my_dummy_model + repository.mapping { indexes :title, analyzer: 'snowball' } + + assert_equal :foobar, DummyRepository.index + assert_equal DummyModel, DummyRepository.klass + assert_equal :my_dummy_model, DummyRepository.type + assert_equal 'snowball', DummyRepository.mappings.to_hash[:my_dummy_model][:properties][:title][:analyzer] + + assert_equal :foobar, repository.index + assert_equal DummyModel, repository.klass + assert_equal :my_dummy_model, repository.type + assert_equal 'snowball', repository.mappings.to_hash[:my_dummy_model][:properties][:title][:analyzer] + end + + should "allow to define gateway methods in the class definition via block passed to the gateway method" do + class DummyRepositoryWithGatewaySerialize + include Elasticsearch::Persistence::Repository + + gateway do + def serialize(document) + 'FAKE!' + end + end + end + + repository = DummyRepositoryWithGatewaySerialize.new + repository.client.transport.logger = Logger.new(STDERR) + + client = mock + client.expects(:index).with do |arguments| + assert_equal('xxx', arguments[:id]) + assert_equal('FAKE!', arguments[:body]) + end + repository.gateway.expects(:client).returns(client) + + repository.gateway.expects(:__get_id_from_document).returns('xxx') + + repository.save( id: '123', foo: 'bar' ) + end + end + + should "allow to define gateway methods in the class definition via regular method definition" do + class DummyRepositoryWithDirectSerialize + include Elasticsearch::Persistence::Repository + + def serialize(document) + 'FAKE IN CLASS!' + end + end + + repository = DummyRepositoryWithDirectSerialize.new + repository.client.transport.logger = Logger.new(STDERR) + + client = mock + client.expects(:index).with do |arguments| + assert_equal('xxx', arguments[:id]) + assert_equal('FAKE IN CLASS!', arguments[:body]) + end + repository.gateway.expects(:client).returns(client) + + repository.gateway.expects(:__get_id_from_document).returns('xxx') + + repository.save( id: '123', foo: 'bar' ) + end + + should "configure the index name in the shortcut initializer" do + assert_equal 'repository', Elasticsearch::Persistence::Repository.new.index_name + end + end +end diff --git a/elasticsearch-persistence/test/unit/repository_naming_test.rb b/elasticsearch-persistence/test/unit/repository_naming_test.rb new file mode 100644 index 000000000..0c6f9833e --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_naming_test.rb @@ -0,0 +1,125 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryNamingTest < Test::Unit::TestCase + context "The repository naming" do + # Fake class for the naming tests + class ::Foobar; end + class ::FooBar; end + module ::Foo; class Bar; end; end + + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Naming }.new + end + + context "get Ruby class from the Elasticsearch type" do + should "get a simple class" do + assert_equal Foobar, subject.__get_klass_from_type('foobar') + end + should "get a camelcased class" do + assert_equal FooBar, subject.__get_klass_from_type('foo_bar') + end + should "get a namespaced class" do + assert_equal Foo::Bar, subject.__get_klass_from_type('foo/bar') + end + should "re-raise a NameError exception" do + assert_raise NameError do + subject.__get_klass_from_type('foobarbazbam') + end + end + end + + context "get Elasticsearch type from the Ruby class" do + should "encode a simple class" do + assert_equal 'foobar', subject.__get_type_from_class(Foobar) + end + should "encode a camelcased class" do + assert_equal 'foo_bar', subject.__get_type_from_class(FooBar) + end + should "encode a namespaced class" do + assert_equal 'foo/bar', subject.__get_type_from_class(Foo::Bar) + end + end + + context "get an ID from the document" do + should "get an ID from Hash" do + assert_equal 1, subject.__get_id_from_document(id: 1) + assert_equal 1, subject.__get_id_from_document(_id: 1) + assert_equal 1, subject.__get_id_from_document('id' => 1) + assert_equal 1, subject.__get_id_from_document('_id' => 1) + end + end + + context "document class name" do + should "be nil by default" do + assert_nil subject.klass + end + + should "be settable" do + subject.klass = Foobar + assert_equal Foobar, subject.klass + end + + should "be settable by DSL" do + subject.klass Foobar + assert_equal Foobar, subject.klass + end + end + + context "index_name" do + should "default to the class name" do + subject.instance_eval do + def self.class + 'FakeRepository' + end + end + + assert_equal 'fake_repository', subject.index_name + end + + should "be settable" do + subject.index_name = 'foobar1' + assert_equal 'foobar1', subject.index_name + + subject.index_name 'foobar2' + assert_equal 'foobar2', subject.index_name + end + + should "be aliased as `index`" do + subject.index_name = 'foobar1' + assert_equal 'foobar1', subject.index + end + + should "be inferred from the host class" do + class ::MySpecialRepository; end + subject.define_singleton_method(:host) { MySpecialRepository } + assert_equal 'my_special_repository', subject.index_name + end + end + + context "document_type" do + should "be nil when no klass is set" do + assert_equal nil, subject.document_type + end + + should "default to klass" do + subject.klass Foobar + assert_equal 'foobar', subject.document_type + end + + should "be aliased as `type`" do + subject.klass Foobar + assert_equal 'foobar', subject.type + end + + should "be settable" do + subject.document_type = 'foobar' + assert_equal 'foobar', subject.document_type + end + + should "be settable by DSL" do + subject.document_type 'foobar' + assert_equal 'foobar', subject.document_type + end + end + end +end diff --git a/elasticsearch-persistence/test/unit/repository_response_results_test.rb b/elasticsearch-persistence/test/unit/repository_response_results_test.rb new file mode 100644 index 000000000..294a96efd --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_response_results_test.rb @@ -0,0 +1,98 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryResponseResultsTest < Test::Unit::TestCase + include Elasticsearch::Persistence + class MyDocument; end + + context "Response results" do + setup do + @repository = Repository.new + + @response = { "took" => 2, + "timed_out" => false, + "_shards" => {"total" => 5, "successful" => 5, "failed" => 0}, + "hits" => + { "total" => 2, + "max_score" => 0.19, + "hits" => + [{"_index" => "my_index", + "_type" => "note", + "_id" => "1", + "_score" => 0.19, + "_source" => {"id" => 1, "title" => "Test 1"}}, + + {"_index" => "my_index", + "_type" => "note", + "_id" => "2", + "_score" => 0.19, + "_source" => {"id" => 2, "title" => "Test 2"}} + ] + } + } + + @shoulda_subject = Repository::Response::Results.new @repository, @response + end + + should "provide the access to the repository" do + assert_instance_of Repository::Class, subject.repository + end + + should "provide the access to the response" do + assert_equal 5, subject.response['_shards']['total'] + end + + should "wrap the response in Hashie::Mash" do + assert_equal 5, subject.response._shards.total + end + + should "return the total" do + assert_equal 2, subject.total + end + + should "return the max_score" do + assert_equal 0.19, subject.max_score + end + + should "delegate methods to results" do + subject.repository + .expects(:deserialize) + .twice + .returns(MyDocument.new) + + assert_equal 2, subject.size + assert_respond_to subject, :each + end + + should "respond to missing" do + assert_instance_of Method, subject.method(:to_a) + end + + should "yield each object with hit" do + @shoulda_subject = Repository::Response::Results.new \ + @repository, + { 'hits' => { 'hits' => [{'_id' => '1', 'foo' => 'bar'}] } } + + subject.repository + .expects(:deserialize) + .returns('FOO') + + subject.each_with_hit do |object, hit| + assert_equal 'FOO', object + assert_equal 'bar', hit.foo + end + end + + should "map objects and hits" do + @shoulda_subject = Repository::Response::Results.new \ + @repository, + { 'hits' => { 'hits' => [{'_id' => '1', 'foo' => 'bar'}] } } + + subject.repository + .expects(:deserialize) + .returns('FOO') + + assert_equal ['FOO---bar'], subject.map_with_hit { |object, hit| "#{object}---#{hit.foo}" } + end + end + +end diff --git a/elasticsearch-persistence/test/unit/repository_search_test.rb b/elasticsearch-persistence/test/unit/repository_search_test.rb new file mode 100644 index 000000000..8229075bb --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_search_test.rb @@ -0,0 +1,82 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositorySearchTest < Test::Unit::TestCase + class MyDocument; end + + context "The repository search" do + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Search }.new + + @client = mock + @shoulda_subject.stubs(:document_type).returns(nil) + @shoulda_subject.stubs(:klass).returns(nil) + @shoulda_subject.stubs(:index_name).returns('test') + @shoulda_subject.stubs(:client).returns(@client) + end + + should "search in type based on klass" do + subject.expects(:klass).returns(MyDocument).at_least_once + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + + @client.expects(:search).with do |arguments| + assert_equal 'test', arguments[:index] + assert_equal 'my_document', arguments[:type] + + assert_equal({foo: 'bar'}, arguments[:body]) + end + + subject.search foo: 'bar' + end + + should "search across all types" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + @client.expects(:search).with do |arguments| + assert_equal 'test', arguments[:index] + assert_equal nil, arguments[:type] + + assert_equal({foo: 'bar'}, arguments[:body]) + end + + assert_instance_of Elasticsearch::Persistence::Repository::Response::Results, + subject.search(foo: 'bar') + end + + should "pass options to the client" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + @client.expects(:search).twice.with do |arguments| + assert_equal 'bambam', arguments[:routing] + end + + assert_instance_of Elasticsearch::Persistence::Repository::Response::Results, + subject.search( {foo: 'bar'}, { routing: 'bambam' } ) + assert_instance_of Elasticsearch::Persistence::Repository::Response::Results, + subject.search( 'foobar', { routing: 'bambam' } ) + end + + should "search with simple search" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + @client.expects(:search).with do |arguments| + assert_equal 'foobar', arguments[:q] + end + + assert_instance_of Elasticsearch::Persistence::Repository::Response::Results, + subject.search('foobar') + end + + should "raise error for incorrect search definitions" do + subject.expects(:klass).returns(nil).at_least_once + subject.expects(:__get_type_from_class).never + + assert_raise ArgumentError do + subject.search 123 + end + end + end + +end diff --git a/elasticsearch-persistence/test/unit/repository_serialize_test.rb b/elasticsearch-persistence/test/unit/repository_serialize_test.rb new file mode 100644 index 000000000..48d0323d8 --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_serialize_test.rb @@ -0,0 +1,57 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositorySerializeTest < Test::Unit::TestCase + context "The repository serialization" do + class DummyDocument + def to_hash + { foo: 'bar' } + end + end + + class MyDocument; end + + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Serialize }.new + end + + context "serialize" do + should "call #to_hash on passed object" do + document = DummyDocument.new + assert_equal( { foo: 'bar' }, subject.serialize(document)) + end + end + + context "deserialize" do + should "get the class name from #klass" do + subject.expects(:klass) + .returns(MyDocument) + + MyDocument.expects(:new) + + subject.deserialize( {} ) + end + + should "get the class name from Elasticsearch _type" do + subject.expects(:klass) + .returns(nil) + + subject.expects(:__get_klass_from_type) + .returns(MyDocument) + + MyDocument.expects(:new) + + subject.deserialize( {} ) + end + + should "create the class instance with _source attributes" do + subject.expects(:klass).returns(nil) + + subject.expects(:__get_klass_from_type).returns(MyDocument) + + MyDocument.expects(:new).with({ 'foo' => 'bar' }) + + subject.deserialize( {'_source' => { 'foo' => 'bar' } } ) + end + end + end +end diff --git a/elasticsearch-persistence/test/unit/repository_store_test.rb b/elasticsearch-persistence/test/unit/repository_store_test.rb new file mode 100644 index 000000000..48b916098 --- /dev/null +++ b/elasticsearch-persistence/test/unit/repository_store_test.rb @@ -0,0 +1,176 @@ +require 'test_helper' + +class Elasticsearch::Persistence::RepositoryStoreTest < Test::Unit::TestCase + context "The repository store" do + class MyDocument; end + + setup do + @shoulda_subject = Class.new() { include Elasticsearch::Persistence::Repository::Store }.new + @shoulda_subject.stubs(:index_name).returns('test') + end + + context "save" do + should "serialize the document, get type from klass and index it" do + subject.expects(:serialize).returns({foo: 'bar'}) + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(MyDocument) + subject.expects(:__get_type_from_class).with(MyDocument).at_least_once.returns('my_document') + subject.expects(:__get_id_from_document).returns('1') + + client = mock + client.expects(:index).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + assert_equal({foo: 'bar'}, arguments[:body]) + end + subject.expects(:client).returns(client) + + subject.save({foo: 'bar'}) + end + + should "serialize the document, get type from document class and index it" do + subject.expects(:serialize).returns({foo: 'bar'}) + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(nil) + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + subject.expects(:__get_id_from_document).returns('1') + + client = mock + client.expects(:index).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + assert_equal({foo: 'bar'}, arguments[:body]) + end + subject.expects(:client).returns(client) + + subject.save(MyDocument.new) + end + + should "serialize the document, get type from document_type and index it" do + subject.expects(:serialize).returns({foo: 'bar'}) + + subject.expects(:document_type).returns('my_document') + + subject.expects(:klass).never + subject.expects(:__get_type_from_class).never + + subject.expects(:__get_id_from_document).returns('1') + + client = mock + client.expects(:index).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + assert_equal({foo: 'bar'}, arguments[:body]) + end + subject.expects(:client).returns(client) + + subject.save(MyDocument.new) + end + + should "pass the options to the client" do + subject.expects(:serialize).returns({foo: 'bar'}) + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(MyDocument) + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + subject.expects(:__get_id_from_document).returns('1') + + client = mock + client.expects(:index).with do |arguments| + assert_equal 'foobarbam', arguments[:index] + assert_equal 'bambam', arguments[:routing] + end + subject.expects(:client).returns(client) + + subject.save({foo: 'bar'}, { index: 'foobarbam', routing: 'bambam' }) + end + end + + context "delete" do + should "get type from klass when passed only ID" do + subject.expects(:serialize).never + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(MyDocument) + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + subject.expects(:__get_id_from_document).never + + client = mock + client.expects(:delete).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + subject.expects(:client).returns(client) + + subject.delete('1') + end + + should "get ID from document and type from klass when passed a document" do + subject.expects(:serialize).returns({id: '1', foo: 'bar'}) + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(MyDocument) + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + subject.expects(:__get_id_from_document).with({id: '1', foo: 'bar'}).returns('1') + + client = mock + client.expects(:delete).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + subject.expects(:client).returns(client) + + subject.delete({id: '1', foo: 'bar'}) + end + + should "get ID from document and type from document_type when passed a document" do + subject.expects(:serialize).returns({id: '1', foo: 'bar'}) + + subject.expects(:document_type).returns('my_document') + + subject.expects(:klass).never + subject.expects(:__get_type_from_class).never + + subject.expects(:__get_id_from_document).with({id: '1', foo: 'bar'}).returns('1') + + client = mock + client.expects(:delete).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + subject.expects(:client).returns(client) + + subject.delete({id: '1', foo: 'bar'}) + end + + should "get ID and type from document when passed a document" do + subject.expects(:serialize).returns({id: '1', foo: 'bar'}) + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(nil) + subject.expects(:__get_type_from_class).with(MyDocument).returns('my_document') + subject.expects(:__get_id_from_document).with({id: '1', foo: 'bar'}).returns('1') + + client = mock + client.expects(:delete).with do |arguments| + assert_equal 'my_document', arguments[:type] + assert_equal '1', arguments[:id] + end + subject.expects(:client).returns(client) + + subject.delete(MyDocument.new) + end + + should "pass the options to the client" do + subject.expects(:document_type).returns(nil) + subject.expects(:klass).at_least_once.returns(MyDocument) + subject.expects(:__get_type_from_class).returns('my_document') + + client = mock + client.expects(:delete).with do |arguments| + assert_equal 'foobarbam', arguments[:index] + assert_equal 'bambam', arguments[:routing] + end + subject.expects(:client).returns(client) + + subject.delete('1', index: 'foobarbam', routing: 'bambam') + end + end + end +end