Computer >> Hướng Dẫn Máy Tính >  >> Lập Trình >> Ruby

Dễ Dàng Test Rails App Trong Docker: Hướng Dẫn Bước Đi

Là Ruby/Rails developers, chúng ta thích viết tests. Testing là phần thiết yếu của software development. Good tests giúp viết high-quality code. Chúng thêm giá trị cho overall development process, nhưng nếu không manage test tốt, chúng làm chậm chúng ta. Dưới đây symptoms của tests improperly managed:

  • Chạy tests mất nhiều thời gian.
  • Tests unreliable và fail ngẫu nhiên.
  • Tests behave khác nhau trên máy khác nhau.
  • Chạy all test suites làm chậm CI.

Trong blog post này, chúng ta cover cách containers dùng để make testing reliable, helpful deliver high-quality software.

Integration Tests Và Unit Tests

Trước khi jump vào containers, hiểu và same page về difference unit và integration tests.

Unit tests test block code in isolation. Chúng test functionality của block code nhưng không test dependencies. Ví dụ, method đơn giản:

def sum(a, b)
  a + b
end

Function sum hai numbers, rất đơn giản. Test function này super easy. Nhưng nếu function có dependency external service. Method phải write to database sau khi complete sum.

def sum!(a, b)
  c = a + b
  Sum.create(value: c)
end

Trong block này, values added và saved to database. Test block này cần database server running. Nếu test với database server running, không còn unit test. Cần external dependency, cũng test database connection và record saved. Đây là integration test.

Unit tests có thể solitary unit tests (mock/stub all dependencies) hoặc sociable unit tests (allow talking to other dependencies). Người ta propose various definitions unit tests. Trong blog context, unit test type chúng ta nói là solitary unit test.

Loại bỏ external dependencies với mocks giúp tests chạy nhanh hơn. Dùng unit tests, leave out dependencies, nhưng để ensure app works as expected, test dependencies equally important. Tests viết test external dependencies như database, network calls, files là integration tests.

Giống unit tests, người ta propose various definitions integration tests. Tuy nhiên basic concept same. Difference chỉ ở scope. Small function write to database có thể integration test. Broader functionality connect multiple parts system cũng integration test. Test ở đây focus narrow part. Càng narrow test, càng nhiều tests cần viết. Với broader tests, số tests required giảm.

Khi chạy integration test, cần application và dependencies như database, external services running. Manage dependencies challenging.

Fundamentals của Docker

Docker là technology giúp abstract application từ environment (operating system) và run nó in isolation với host. Giống virtual machine nhưng nhanh và efficient hơn. Docker cho phép package application code và dependencies vào single container. Giúp code portable và same Docker image running development, testing, production. Đảm bảo consistency environment app runs. Vì không dependent host server, làm application behavior predictable all environments. Docker giúp developers run và manage application và environment, bao gồm building, running, deploying, updating dùng simple commands.

Hiểu terminologies quan trọng:

  • Dockerfile - File định nghĩa dependencies required cho application. Chỉ định OS, database client libraries, packages required. Cũng define command required app run. Dockerfile start từ base image. Base images có thể OS hoặc programming language.

  • Images - Blueprint states steps required container run. Images built với different layers. Every step Dockerfile là layer trong image. Tăng reusability và helps caching intermediate layer build image faster.

  • Containers - Actual images. Có thể stopped state. Containers started từ docker image. Single docker image start multiple docker containers. Containers hold information specified by image. Có thin container layer lưu changes made to image while app running.

  • Docker Compose - Tool khác manage multiple containers. Có multiple containers từ same docker image. Cũng cần multiple containers từ different docker images, containers cần interact. Docker Compose manage containers. Tools khác available, nhưng context này chỉ dùng Docker Compose vì simplicity.

Virtual Machines trên host hypervisor cần separate OS installed, dùng more host resources. Với Docker, chỉ cần install Docker engine host OS, host resource management by Docker. Docker làm life easier developers và sysops, popular DevOps culture.

Tại Sao Dùng Docker Cho Testing

Docker isolate running application hoặc test từ host environment với minimal resource consumption. Advantages dùng docker testing:

  • Lightweight: Docker containers lightweight. Giúp run multiple containers same time không one test process cause issue another test process.

  • Portable: Docker images portable run any environment. Giúp CI, reuse same image run tests local và CI environments.

  • Versioning: Multiple applications hoặc upgrade dependencies yêu cầu different versions dependencies local machine. Rất difficult manage không Docker. Với Docker, different containers running different version. Cũng run tests different versions verify things.

  • Easy management microservices: Microservices, testing distributed và complicated. Cần manage dependent microservice. Docker và Docker Compose help manage service dependencies.

Pre-requisites

Để try this, ensure sau setup:

  • Docker
  • Docker-compose
  • Ruby 2.7 (Optional, sẽ add Ruby to Docker container, nhưng helpful testing)
  • Rails (5.1+ recommended)

Dockerize App

Bắt đầu tạo simple Rails app và containerize. Tôi start fresh Rails app. Bạn cũng containerize existing Rails app.

rails new calculator

Tạo file Dockerfile với content:

FROM ruby:2.7
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update -qq && apt-get install -y nodejs yarn
RUN mkdir /calculator
WORKDIR /calculator
COPY Gemfile /calculator/Gemfile
COPY Gemfile.lock /calculator/Gemfile.lock
RUN bundle install
COPY package.json /calculator/package.json
COPY yarn.lock /calculator/yarn.lock
RUN yarn install --check-files
COPY . /calculator
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]

Dockerfile defined có Ruby 2.7 base image. Cần NodeJS và Yarn, next step install chúng. Sau installed, chạy bundle install và yarn install. Cuối copy all code và run Rails server với port 3000 exported. Dockerfile rất đơn giản. Với này, Rails app dockerized.

Giờ create và run docker image đảm bảo running.

Build docker image:

docker build . -t calculator

Run container:

docker run -p 3000:3000 calculator

Tạo Model Với Calculation Logic

Tạo model calculation add calculation logic. Dùng model viết logic và tests. Xem manage dependencies tests dùng Docker Compose.

Hiện tại chạy Rails với SQLite. Đổi sang Postgres thay SQLite. Dễ dàng command:

rails db:system:change --to=postgresql

Generate calculation model:

bundle exec rails generate model calculation

Add fields calculation schema:

# db/migrate/create_calculation.rb
class CreateCalculations < ActiveRecord::Migration[6.0]
  def change
    create_table :calculations do |t|
      t.integer :x
      t.integer :y
      t.integer :result
      t.timestamps
    end
  end
end

Thử chạy database migration trong container:

docker-compose exec web rails db:migrate

Vì database không chạy trong container, command throw error.

rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Chạy Postgres image as another image add network giữa chúng. Manage container communication dùng Docker Compose.

Dùng Docker Compose Quản Lý Multiple Containers

Với Rails app containerized, add dependencies to service dùng Docker Compose. Dependency app là Postgres.

Postgres Docker image available publicly, không cần tạo. Tuy nhiên cần manage database container và Rails app container.

version: "3.0"
services:
  db:
    image: postgres
    environment:
      POSTGRES_USER: calculator
      POSTGRES_PASSWORD: password
      POSTGRES_DB: calculator_test
  web:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - db
    environment:
      CALCULATOR_DATABASE_PASSWORD: password

Giờ vào database.yaml đổi database host khác. Host sẽ là name specified inside service Docker Compose file. Case này host db. Đảm bảo DB password same trong service postgres và web. Cũng đổi username, password, database name như Docker Compose file.

# database.yaml
development:
  <<: *default
  database: calculator_development
  username: calculator
  password: <%= ENV['CALCULATOR_DATABASE_PASSWORD'] %>

Giờ start containers single command:

docker-compose up

Start cả database và Rails server. Chạy migration command, connect Postgres và run migration.

Thêm Tests

Setup ready, viết simple method trong model và test method:

# calculation.rb
class Calculation < ApplicationRecord
  def self.sum(x, y)
    result = x + y
    Calculation.create(x: x, y: y, result: result)
    result
  end
end

Sum method không chỉ add numbers mà save numbers to database. Test method yêu cầu database instance running.

Test này integration test, vì must connect database.

Dùng Mini-test viết test, Rails default.

# calculation_test.rb
require 'test_helper'
class CalculationTest < ActiveSupport::TestCase
  test "should sum and save the data" do
    result = Calculation.sum(1, 2)
    c = Calculation.last
    assert result, 3
    assert c.result, result
  end
end

Chạy test:

docker-compose exec web rails test

Test trên test sum method add values và save values database. Với Docker Compose, easy way connect database.

Method dependent database. Dependency không chỉ database mà third-party service cung cấp REST API. Thử dùng third-party service cung cấp sum method thay vì tự viết.

# calculation.rb
class Calculation < ApplicationRecord
  def self.sum(x, y)
    # Hostname same as specified in docker-compose file
    url = 'https://sumservice:4010/sum'
    uri = URI(url)
    params = { :x => x, :y => y }
    uri.query = URI.encode_www_form(params)
    res = Net::HTTP.get_response(uri)
    throw "Error" unless res.is_a?(Net::HTTPSuccess)
    result = res.body.to_i
    Calculation.create(x: x, y: y, result: result)
    result
  end
end

Làm tương tự REST API như database dependency. Cần ensure valid endpoints, nhưng không muốn call actual service during test. Case này tạo mock/stub tests, nhưng tests không integration test nếu làm vậy. Muốn test realistic as possible. Tạo mock API endpoint, chạy trong container, call mock container API service, respond result.

Sửa test trước:

# calculation_test.rb
require 'test_helper'
class CalculationTest < ActiveSupport::TestCase
  test "should sum and save the data" do
    result = Calculation.sum(1, 2)
    c = Calculation.last
    # we don't need this assetion as we are deligation this responsibility to external service:
    # assert result, 3 // Not needed
    assert c.result, result
  end
end

Sửa test case chỉ test result provided by sum API saved properly database. Không cần test actual sum value correct, vì handled by external service, trust them.

Generate mock server, dùng openAPI. Tạo folder mock root directory. Bên trong tạo file api.yaml

# api.yaml
swagger: "2.0"
info:
  description: "This is a sum api provider"
  version: "1.0.0"
  title: "API Provider"
  host: "https://mock-service.com/"
  basePath: "/api"
  schemes:
    - "https"
    - "http"
paths:
  /sum:
    get:
      produces:
        - "application/json"
      parameters:
        - name: "x"
          in: "query"
          description: "x value"
          required: true
          type: "integer"
        - name: "y"
          in: "query"
          description: "y value"
          required: true
          type: "integer"
      responses:
        "200":
          description: "successful operation"
          schema:
            type: "integer"
            example: 3

Chạy server based on specification, dùng mock service provider. Tôi dùng tool prism. Có docker image, add docker-compose file:

version: "3.0"
services:
  db:
    image: postgres
    environment:
      POSTGRES_USER: calculator
      POSTGRES_PASSWORD: password
      POSTGRES_DB: calculator_test
  web:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - db
      - sumservice
    environment:
      CALCULATOR_DATABASE_PASSWORD: password
  sumservice:
    image: stoplight/prism:4
    command: mock -h 0.0.0.0 "/tmp/api.yaml"
    volumes:
      - ./mock:/tmp/
    ports:
      - 4010:4010
    init: true

Chạy test xem working:

> docker-compose exec web rails test
Finished in 0.337710s, 2.9611 runs/s, 5.9222 assertions/s.
1 runs, 2 assertions, 0 failures, 0 errors, 0 skips

Yeah! Test running. Method connect sum service Docker image get sum và connect Postgres save data. All dependencies managed by Docker Compose, không lo dependencies installed previously. Move test any machine work without dependency install. Chỉ cần machine có docker.

Nếu app depend other dependencies như Redis hoặc Memcached, tìm dockerhub. Add images vào Docker Compose file as needed.

Parallel Tests

Rails 5.1.5+ support chạy tests parallel. Default, tests run parallel từ Rails 6 based on number of processors. Khi chạy test, tạo database instances based on number parallel tests. Nếu 4 processors, tạo databases "calculator_test-0", calculator_test-1, calculator_test-2, calculator_test-3. Vào postgres cli check databases:

> \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-------------------+------------+----------+------------+------------+---------------------------
calculator_test | calculator | UTF8 | en_US.utf8 | en_US.utf8 |
calculator_test-0 | calculator | UTF8 | en_US.utf8 | en_US.utf8 |
calculator_test-1 | calculator | UTF8 | en_US.utf8 | en_US.utf8 |
calculator_test-2 | calculator | UTF8 | en_US.utf8 | en_US.utf8 |
calculator_test-3 | calculator | UTF8 | en_US.utf8 | en_US.utf8 |

Giờ, even not using Rails hoặc mini-tests hoặc older Rails, vẫn run tests parallel nếu viết với Docker. Rails tự tạo databases parallel tests, nhưng nếu other dependencies như Redis hoặc memcached, có thể manage manually.

Vì all dependencies managed, chạy Docker Compose với different arguments run tests parallel. Mục đích, specify folders run tests parallel, execute create multiple Docker Compose files, run parallel.

Thêm Makefile Đơn Giản Commands

Vì dùng Docker và Docker Compose commands chạy tests, commands dài. Khó nhớ và chạy daily development. Đơn giản simple command, dùng Makefile. Makefile simple text file reference target command execution.

.PHONY: test
up:
  docker-compose up
down:
  docker-compose down
test:
  docker-compose exec web rails test

Thêm vài commands makefile. Extend as required by application. Add makefile root directory Rails application. Khi có makefile, chạy test simple command:

Boot containers:

> make up

Run test:

> make test

Dùng Docker Cho Local Development

Cũng dùng Docker develop application. Giúp ensure application behaves consistently across testing, development, production. Review cần change gì dùng container cho local development.

Volume Mount

Khi chạy app locally, muốn app reflect changes local server khi make changes code. Current scenario, code copied từ machine to Docker, server started. Nếu make changes file local machine, không reflect trong Docker. Đảm bảo code always in sync, dùng volume mount.

Trong Docker Compose file, add volumes:

version: "3.0"
services:
  db:
    image: postgres
    environment:
      POSTGRES_USER: calculator
      POSTGRES_PASSWORD: password
      POSTGRES_DB: calculator_test
  web:
    build: .
    volumes:
      - .:/calculator  # Volume mounted
    ports:
      - "3000:3000"
    depends_on:
      - db
      - sumservice
    environment:
      CALCULATOR_DATABASE_PASSWORD: password
  sumservice:
    image: stoplight/prism:4
    command: mock -h 0.0.0.0 "/tmp/api.yaml"
    volumes:
      - ./mock:/tmp/
    ports:
      - 4010:4010
    init: true

Giờ, make changes anything sever, changes reflected. Thử tạo controller call sum method:

# controllers/calculation_controller.rb
class CalculationController < ApplicationController
  def index
    result = Calculation.sum(params[:x], params[:y])
    render json: {result: result}
  end
end

Add root route:

# routes.rb
Rails.application.routes.draw do
  # For details on the DSL available within this file, see https://guides.rubyonrails.org/routing.html
  root "calculation#index"
end

Giờ quay lại check app browser, expected response:

Dễ Dàng Test Rails App Trong Docker: Hướng Dẫn Bước Đi

Lưu ý start server dùng mock sumservice, bất kể query parameter URL, luôn return 2. Không似好 development nhưng mock service provider có advantages. Develop microservices, nhiều dependent services, cần boot up trước start service. Tăng productivity development. Không concerned other service, chỉ service developing.

Muốn final verification how works với actual service, tạo new Docker Compose file replace actual image service. Nếu internal service, replace latest stable Docker image. Khuyên có mock service even development environment.

.dockerignore

Khi app larger, dependencies cause Docker build context larger. Larger build context, more time build docker image. Volume mount cũng cause delay với increased size app. Case này, add unwanted files .dockerignore reduce build context.

.git*
log/*
README.md
node_modules

Entrypoint

Khi Rails server running container, sometimes error server already running vì PID file not deleted khi container exited. Remove error, add ENTRYPOINT remove server.pid file.

FROM ruby:2.7
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update -qq && apt-get install -y nodejs yarn
RUN mkdir /calculator
WORKDIR /calculator
COPY Gemfile /calculator/Gemfile
COPY Gemfile.lock /calculator/Gemfile.lock
RUN bundle install
COPY package.json /calculator/package.json
COPY yarn.lock /calculator/yarn.lock
RUN yarn install --check-files
COPY . /calculator
EXPOSE 3000
ENTRYPOINT ./entrypoint.sh
CMD ["rails", "server", "-b", "0.0.0.0"]

Tạo file entrypoint.sh:

#!/bin/bash
set -e
# Remove a potentially pre-existing server.pid for Rails.
rm -f /calculator/tmp/pids/server.pid
# Exec the container's main process (CMD command in rails file)
exec "$@"

Make executable

chmod +x entrypoint.sh

Giờ chạy Docker image, không experience issue này.

Kết Luận

Rất quan trọng distinguish integration tests và unit tests. Viết unit tests, mock external dependencies như databases. Giúp test code isolation. Cũng equally important run integration tests giữa core business logic và dependencies. Dependencies như databases và external services test integration dùng Docker và Docker Compose. Good practice separate hai loại. Nên run separately. More unit tests và fewer integration tests makes tests run faster và reliably.

Advantages workflow:

  • Segregation giữa unit và integration tests.
  • Vì all CI services run on containers, tests running locally và CI same behavior. Ensure consistency test behavior local machine và CI server.
  • Dùng containers deploy application cũng ensure consistency production server.
  • Reliable tests, dependencies well managed.
  • Portable tests, little effort setup khi multiple developers dùng.