<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=381191815628942&amp;ev=PageView&amp;noscript=1">

Cramming for the test not encouraged

By Tom Hickman, VP of Engineering — Dec 05, 2017

DevOps promises efficiency, low cycle time, and fast time-to-value. For that to work, product qualities—functional correctness, performance, resiliency, security—have to be built-in. You can’t “season to taste” by sprinkling some performance or security on your product like you might add salt and pepper to your beef stew.

Salt and pepper notwithstanding, it’s crazy busy in the kitchen these days. It’s all small batch size, rapid iteration, continuous integration, and automated deployment.

Gone are the bad old days of software development. Gone the three-year project. Gone the three hundred page system spec document. Gone too is the hardening sprint! Put differently, you can’t cram for the test when it comes to software qualities.

Getting from then to now hasn’t been easy, and it hasn’t been fast. Test-Driven Development (TDD) has been around since the late 90s, but I still don’t know many developers who actually write their tests before they write their code.

On the plus side, however, I also don’t know many developers who still argue against TDD as a best practice. And unit testing—once a radical concept and purely a nice-to-have—is now part of nearly every shop’s “definition of done.”

Functional correctness is not deferrable. You cannot make a product right by testing alone. You can only harden its code paths to the tests you think of.

Testing in depth is part of the development process. Test code is product code. The layers in the test pyramid—unit tests, contract tests, integration tests, performance tests, soak tests—are as important as the successive pipeline environments leading eventually to production.

Performance is not deferrable. You cannot make your product scale to your targets—load, concurrent sessions, or data throughput—by testing alone (click to tweet this). With testing, you can only predict when and how your product will fail.

And of course, amongst all these product qualities, security is similarly non-deferrable. The software we make today is under constant threat, and very probably constant attack. Just ask anyone who’s had their product pwned. A security hardening sprint simply won’t work. The DevSecOps movement recognizes this, adopts a posture of security-in-depth, and makes sure that security is never an afterthought.

The long arc of history (well… software development history) has led inexorably to the modern DevOps movement: small change sets, rapid cycle time, and frequent deployments (click to tweet this).

If you’re good at DevOps, you already believe that you you can’t test your way to functional correctness, performance, security or any other product quality.

If you’re a DevSecOps practitioner, you don’t rely on your annual pen test, your SAST or DAST provider, or any other single step in your pipeline to ensure or assure your product quality. You build security in. You train your developers. You harden your perimeter. You implement a security-in-depth approach, and you do not try to cram for the final in the last few days before you deploy to production. At Edgewise Networks, we’re working on making that last bit easier, for our customers and ourselves. We’re doing that while also practicing what we preach, building in product qualities—usability, performance, functional correctness, scalability, security—well upstream of the “deploy to prod” step.


Tom Hickman, VP of Engineering

Written by Tom Hickman, VP of Engineering

Tom is VP of Engineering at Edgewise, which marks his eighth startup. Past roles have included Director of Global Sourcing at Iron Mountain where he built and maintained a global outsourcing center of excellence, and Vice President of Engineering at My Perfect Gig, an agile development firm that built data-filled search and analytic software for the technology recruiting market. Most recently, Hickman served as the Vice President of Engineering at Veracode where he led engineering and product strategy, helping to grow Veracode from a single product company to a multi-product security platform that was recently acquired by CA Technologies for more than $600 million.