Writing Tests for Plugins

Rspec goal: 80% Coverage

I’d like to keep tackling rspec tests until we can reach at least 80% coverage on each of our respective plugins. On the coverage % front, Discourse has a useful gem called “simplecov” which will calculate the percentage of your code that is covered by tests based on what lines are touched in the course of a test suite. I’ve found that you can restrict SimpleCov to a plugin by adding the following to your spec/plugin_helper.rb file:

SimpleCov.configure do
  add_filter do |src|
    src.filename !~ /discourse-custom-wizard/ ||
    src.filename =~ /spec/

Once this is added you run your plugin test suite like this to get a coverage report for your plugin:

bundle exec rake plugin:spec[plugin_name] COVERAGE=1

You’ll then see an additional line of output at the end of the tests:

Coverage report generated for RSpec to /Users/angusmcleod/discourse/discourse/coverage. 589 / 1213 LOC (48.56%) covered.

If you look at the coverage report, it’ll tell you exactly what lines of code are not being tested.


I got a head start on this today by writing up rspec and qunit tests for the multilingual plugin I’m building for Wikimedia.

Some tips

  1. You should require the spec/rails_helper at the top of each spec file

    require 'rails_helper'

    This will load all the necessary stuff for rspec testing, including whatever you put in plugin/spec/plugin_helper.rb.

    This means plugin_helper.rb is a good spot to put rspec global config. For example I wrapped all tests in a code block to ensure that non-local language data was properly loaded for each test, see: discourse-multilingual/plugin_helper.rb at main · paviliondev/discourse-multilingual · GitHub

  2. It’s easier to run qunit tests in the browser directly, than via the command line, e.g.

  3. For those who already have a .travis.yml file, e.g. events and tlp, this probably needs to be updated in line with the instructions here: (Superseded) Set up plugin continuous integration tests on Travis CI - developers - Discourse Meta.

  4. You may find that data persists between tests, particularly data in the PluginStore. To ensure this doesn’t happen you can add this to your plugin_helper.rb file

RSpec.configure do |config|
  config.around(:each) do |example|
    ActiveRecord::Base.transaction do
      raise ActiveRecord::Rollback

Why might I be getting 0/0 for my coverage report?

merefield@scw-40efa1:~/discourse$ RAILS_ENV=test bundle exec rake plugin:spec[discourse-follow] COVERAGE=1
LOAD_PLUGINS=1 /home/merefield/.rbenv/versions/2.6.3/bin/ruby -S rspec ./plugins/discourse-follow/spec/lib/updater_spec.rb --profile

Randomized with seed 35452

Top 3 slowest examples (0.32995 seconds, 34.2% of total time):
  Follow::Updater expect users followers to include follower
    0.13899 seconds ./plugins/discourse-follow/spec/lib/updater_spec.rb:8
  Follow::Updater expect users followers to include multiple followers
    0.1113 seconds ./plugins/discourse-follow/spec/lib/updater_spec.rb:14
  Follow::Updater sent a notification
    0.07965 seconds ./plugins/discourse-follow/spec/lib/updater_spec.rb:22

Finished in 0.96455 seconds (files took 9.02 seconds to load)
3 examples, 0 failures

Randomized with seed 35452

Coverage report generated for RSpec to /home/merefield/discourse/coverage. 0 / 0 LOC (100.0%) covered.

Branch here:


You shouldn’t be running it in the test env. Use

bundle exec rake "plugin:spec[discourse-follow]" COVERAGE=1

Bizarrely that makes no difference, still getting:

Coverage report generated for RSpec to /home/merefield/discourse/coverage. 0 / 0 LOC (100.0%) covered.

The issue seems to be that simplecov isn’t detecting your tests (i.e. 0 Lines)

Perhaps you’ve added a filter in your plugin_helper.rb that’s filtering out all the plugin paths?

add_filter do |src|
    src.filename !~ /discourse-custom-wizard/ ||
    src.filename =~ /spec/ ||
    src.filename =~ /db/ ||
    src.filename =~ /api/ ## API features are currently experimental

Something fishy with that pattern.

If instead I use:

SimpleCov.configure do
  add_filter do |src|
    src.filename =~ /discourse-follow/ &&
    src.filename =~ /spec/

I get:

Coverage report generated for RSpec to /home/merefield/discourse/coverage. 341 / 1371 LOC (24.87%) covered

Which seems about right to me?


src.filename =~ /discourse-follow/ &&
src.filename =~ /spec/

means filter out any file that includes BOTH discourse-follow and spec in its path. So it’s a bit broad, as nothing really will be filtered, apart from the follow specs. It will include files outside of the plugin. If you then look at the coverage report itself (it will appear as an index.html in a “coverage” folder in the root discourse foloder), we can see the actual files it’s covering

Screen Shot 2020-11-17 at 10.20.07 AM

Your previous version

add_filter do |src|
  src.filename !~ /discourse-follow/ ||
  src.filename =~ /spec/

Actually works for me (I pulled your tests branch)

Screen Shot 2020-11-17 at 10.15.43 AM

I’m getting a 64% figure.

Now if we look at the coverage report index.html it’s only covering the plugin files

Screen Shot 2020-11-17 at 10.18.08 AM

Super odd. I’ll take another look.

On the plus side: getting close :wink:

OK, i’m getting the same problem with two other plugins (topic-previews and custom-wizard) so it’s something to do with my dev setup …

@angus This is driving me nuts.

  • I’ve rolled back to match your version of Ruby (2.6.1).
  • I’m on latest master

Still I get 0/0 for any plugin coverage.

Just for posterity, Rob and I had a 1.5hr call to try and debug this, but still no joy. If anyone has any experience with a similar simplecov issue, any tips would be most appreciated.

@merefield As we discussed, let’s move on for now, but it might be worth posting on meta too.