Skip to content

WIP: Add results to jobs

Dan Davison requested to merge job-results into master

What does this MR do?

Add results to jobs.

A headache that we face in Quality is the lack of an integrated test-reporting mechanism.

This MR adds a result model nested under jobs that allows a generic way for users to report any result from said jobs.

Routes:

/*namespace_id/:project_id/-/jobs/:id/results(.:format)         
/*namespace_id/:project_id/-/jobs/:id/results(.:format)         
/*namespace_id/:project_id/-/jobs/:id/results/new(.:format)     
/*namespace_id/:project_id/-/jobs/:id/results/:id/edit(.:format)
/*namespace_id/:project_id/-/jobs/:id/results/:id(.:format)     
/*namespace_id/:project_id/-/jobs/:id/results/:id(.:format)     
/*namespace_id/:project_id/-/jobs/:id/results/:id(.:format)     
/*namespace_id/:project_id/-/jobs/:id/results/:id(.:format)     

As far as Quality Dogfooding - when one of our QA specs finish running, we will report the result of the specs in the job. POST /api/v4/projects/:id/jobs/:job_id/results

Then we could fetch results based on the pipeline runs:

GET /api/v4/projects/:id/pipelines/:pipeline_id/results

[
  {
    "id": 1,
    "name": "qa/specs/features/browser_ui/3_create/merge_request/create_merge_request_spec.rb",
    "result": "passed",
    "field1": null,
    "field2": null,
    "field3": null,
    "field4": null,
    "field5": null,
    "field6": null,
    "field7": null,
    "field8": null,
    "field9": null,
    "field10": null
  },
  {
    "id": 2,
    "name": "qa/specs/features/browser_ui/2_plan/issue/create_issue_spec.rb",
    "result": "failed",
    "field1": "Failure/Error: expect(bar_component).to have_detailed_metrics
        expected #has_detailed_metrics? to return true, got false
      # ./qa/specs/features/browser_ui/non_devops/performance_bar_spec.rb:27:in `block (4 levels) in <module:QA>'
      # ./qa/scenario/actable.rb:16:in `perform'
      # ./qa/specs/features/browser_ui/non_devops/performance_bar_spec.rb:25:in `block (3 levels) in <module:QA>'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:123:in `block in run'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `loop'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `run'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
      # ./spec/spec_helper.rb:68:in `block (2 levels) in <top (required)>'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:123:in `block in run'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `loop'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `run'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
      # /usr/local/bundle/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
      # ./qa/specs/runner.rb:69:in `perform'
      # ./qa/scenario/template.rb:10:in `block in perform'
      # ./qa/scenario/template.rb:8:in `tap'
      # ./qa/scenario/template.rb:8:in `perform'
      # ./qa/scenario/template.rb:33:in `perform'
      # ./qa/scenario/template.rb:10:in `block in perform'
      # ./qa/scenario/template.rb:8:in `tap'
      # ./qa/scenario/template.rb:8:in `perform'
      # ./qa/scenario/bootable.rb:28:in `launch!'
",
    "field2": null,
    "field3": null,
    "field4": null,
    "field5": null,
    "field6": null,
    "field7": null,
    "field8": null,
    "field9": null,
    "field10": null
  }
]

Screenshots

Screen_Shot_2019-12-17_at_6.51.07_PM

Screen_Shot_2019-12-17_at_6.51.46_PM

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by 🤖 GitLab Bot 🤖

Merge request reports