If you are new to BDD/TDD/Specflow/Cucumber/Gherkin buzz, its worth spending few minutes reading this. Intro to BDD
How about working with team from business that created 100+ Specflow feature scenarios before we even start writing a single C# code! It’s astonishing, TDD in its own form, and I love it! So we supported all behaviours and make them all passed over time. Deployed the services. Done.
As we get better, we receive more feature scenarios: like 5 today, and maybe 10 more next week. To support this, we need to continously recompile the test project and sometimes only to realize the tests failed due to faulty feature files or bad test data. This feedback loop has been working but could certainly be better.
What if our analysts can:
- Create new or modify existing feature files on his/her computer
Run tests by issuing commands like these
# execute a specific feature / SpecRunner.exe /feature: FoobarFeatureFile.feature /assembly: FoobarScenarioBindings.dll
# execute all features from a drop folder / SpecRunner.exe /feature: C:/FoobarFeatures /assembly: FoobarScenarioBindings.dll
Get a report and start discussions with dev team
Solution Demo (3-mins, best viewed in 1080p HD)
This vid shows a PoC where I used Specflow and xUnitAdapter to execute external feature files. The first run executes all embedded feature files in the DLL. Next run, I added a feature file and re-executed the tests. No recompile, runs on console.
It’s inspired by Gaspar Nagy’s (Specflow creator) sample project for the xUnitAdapter plugin for VS. I thought it may just work out well with a console runner.
- VS 2017 + Specflow for VS2017
- .NET Framework 4.5.2 Developer Pack
On a fresh Specflow project in VS2017, add feature files and run the ff steps:
- Install Specflow xUnit adapter
- Install xUnit console runner
- Remove all code behind files from
Compile and run from console
/ cd C:\play\docker-sql-sandbox\specflow\packages\xunit.runner.console.2.3.1\tools\net452 / xunit.console.exe "C:\play\docker-sql-sandbox\specflow\SpecflowTests\bin\Debug\SpecflowTests.dll" -html c:\temp\specflow-test-results.html
Add more feature files in the
ExternalFeaturesfolder and re-run. Check out if new feature files are executed.
Alternatively, checkout full source code here: https://github.com/rdagumampan/docker-specflow
- The adapter only works with xUnit.
- Analysts uploads into repo. Auto-trigger CI/CD and run Specflow regression tests (as-is).
- Analysts runs in local machine, fixed local, deliver to dev when its good enough (preferred, but we keep #1).
In real-world, these regression tests cannot run on its own. It needs all the interacting services and datastores. So I will create a portable container containing all the services and datastore that data analysts can orchestrate on their own while making more feature files or refinining the models to suite the tests scenarios. Watch out :)
Gaspar Nagy (Specflow creator) shares his story behind the Specflow code-behind