Nâng Cao Production Debugging Với Breadcrumbs

Bạn từng deal với error production, và dù thử gì, không replicate issue trên development/staging? Next step thường gather more data bằng debug log production. Nếu không có good way correlate logs với request, frustrating, especially during incident.

Chúng tôi thêm feature giúp, gọi là Breadcrumbs.

Breadcrumb rất giống log event, nhưng stored và reported alongside error. Giống log, Breadcrumb chứa message, nhưng cũng hold metadata (hashmap). Set breadcrumbs collected throughout request life (hoặc job invocation), immediately dropped trừ khi error reported.

Add own breadcrumbs simple. Call Honeybadger.add_breadcrumb anywhere code:

Honeybadger.add_breadcrumb("Loading User", metadata: { user_name: user_name })

Nếu error reported sau đó, thấy trong Breadcrumb stack:

Ôi user_name trông empty, có thể cause problems.

Automatic Breadcrumbs

Honeybadger Ruby library có hooks vào Ruby & Rails tự động collect breadcrumbs. Ví dụ, all log messages emitted Production (sent through Logger class) captured và created as breadcrumbs. Cũng tie vào Rails Instrumentation gather breadcrumbs cho Controller Actions, SQL Queries, Active Job invocations, etc...

How Can You Use It?

Breadcrumbs available via Ruby client. Update version 4.4.0 và enable trong config vì disabled by default cho đến release 5.0.0 gem, ensure work out kinks.

Thử và cho biết nếu issues.

Extending Breadcrumbs

Breadcrumbs formally introduced, xem quick example extend. Enable Breadcrumbs Rails app, get standard instrumentation breadcrumbs free, nhưng muốn more?

Giả sử muốn Breadcrumb mỗi lần app send HTTP request. Info handy debugging.

Simple way: call Honeybadger.add_breadcrumb at each request invocation.

def send_a_message
  res = conn.post("/message", { user: user.id, body: "Hey!" }.to_json)
  Honeybadger.add_breadcrumb("Request: /message", metadata: { user: user.id })
  res
end

Store breadcrumb sau mỗi POST /message. Lần sau app throw error sau send message, thấy user nào sent và khi nào relative to error, Yay!

Hơi cumbersome, cần add_breadcrumb tại mỗi location send request. Note, however, are advantages tạo breadcrumbs như này. Specific về metadata capture. Often great way gather targeted info motivated by production bug.

Thực sự chỉ muốn biết khi nào và đâu request goes out. Nice nếu library làm most work ;).

Instrumenting With Faraday

Cheat một chút assume dùng Faraday request library phổ biến.

Could build own middleware, nhưng dùng Rails instrumentation as broker. May mắn faraday_middleware gem cung cấp middleware emit events. Ensure gem in Gemfile và instrumentation middleware injected vào connection.

connection = Faraday.new do |conn|
  conn.use :instrumentation
  conn.adapter Faraday.default_adapter
end

Có thể require work nếu không share connection inside app. Nếu lazy cũng prepend ConnectionOptions ensure all requests có middleware enabled. Careful, sẽ instrument ANY Faraday requests trong app và included gems (có thể want)!

Rails instrumentation nice side effects: multiple subscriptions cho other use cases (general logging).

Binge Watching Our Requests

Faraday requests instrumented, subscribe tạo breadcrumbs:

ActiveSupport::Notifications.subscribe "request.faraday" do |_, _, _, _, data|
  method = data[:method].to_s.upcase
  metadata = data.to_h.slice(:url, :status).merge({method: method})
  Honeybadger.add_breadcrumb("#{method}: #{metadata[:url]}", category: "request", metadata: metadata)
end

Honeybadger now adding breadcrumb cho each outgoing request!

Notice chỉ inspect few data points. Không add outgoing/response body payloads vì chance sensitive data leak vào breadcrumb metadata.

Let Us Know How It Goes!

Hy vọng Breadcrumbs helpful addition debugging toolbox. Try out, shout nếu muốn thêm gì.