Why does my firmware need … a Quality Process? (2)

In my last post, I said I couldn’t write your Quality Process for you, and I can’t. That’s because your process needs to be personalised (or ‘teamalised’, even though it isn’t a proper word). Only you and your team know how you work, which is the core content of your QP document. But there is guidance, and there are hints, that I can offer.

Never ever include unnecessary detail. [This applies to all technical documents.] This guideline benefits you at least as much as it benefits anyone or anything else. There are a surprising number of justifications for this. Here are a few of them.

  • If you commit yourself to producing a design that is green (I know; just trying to keep my analogies simple), then find during coding (which is also design) that it needs to be blue, you have to go back and change your document too. Just say the design will be coloured to optimise reflectivity (or whatever), or don’t mention colour at all, unless it’s a significant attribute of your design.
  • If you specify too much at any stage or abstract level, you risk doing too much up front, (unnecessarily) constraining the design that still remains to be done.
  • Unnecessary detail takes attention away from the significant things you have to say.
  • Unnecessary detail often breaks DRY, which is something we never do. 😉

Write your QP for trusted firmware designers, not for miscreants that must be kept in order. We are all professionals; mutual respect is essential.

Never add anything to your QP that isn’t true. It will be no good to you if you do. But when you write it down for the first time, you may see that the way you work is not how you think you should work. Good! Agree the right way to work with all involved parties, change over to the new practice, and then describe it in your QP. Your QP has already benefitted your work.

Every firmware project is subject to requirements that change (or are added) during its lifetime. The development process you describe should describe how you deal with this. Agile techniques offer possible answers; I’m sure there are others.

If you always spend considerable time debugging, include that in your QP description. [Then, perhaps, you might want to give some thought about how you can spend more time on the good stuff, and less on debugging. 🙂 Dare I suggest TDD?] If you carry out, or assist in, acceptance testing, include that too. Don’t miss out any contribution you make to the development process.

When you have finished, you will have a checklist, useful to the newbie and the experienced developer alike. It will be especially helpful if you are unlucky enough to work without the support of a team. It will help your managers too.

Hope this helps?

This is an addendum to the fourth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … a Quality Process? (2)

Why does my firmware need … software patterns?

Our firmware needs software patterns as much as any software does. Which is to say we don’t need them, but they are a valuable tool. A pattern is written by a designer experienced enough to have come across many specific problems, and recognise them as instances of one general problem. Then it offers a general solution that has worked well, explaining why it might suit your specific problem, and (more importantly?) why it might not. A pattern is a valuable capsule of design information.

As a firmware designer, you will find that most (not all!) published patterns describe problems that you do not, and will not, encounter. [This applies to all users of patterns.] Inasmuch as anyone is to blame, this is our fault. It takes seasoned firmware designers to recognise and describe the problems that occur commonly in our work. If we want more firmware patterns, we need to write them…. [There are a couple of books — here and here — that look promising; I haven’t read them. If you have, please let us know what you thought of them. Thanks.]

The solutions recommended in patterns are dependent on various things. The implementation language is one of them. A pattern that is useful for one language might be unnecessary in another, whose native features automatically solve the problem, or prevent it from occurring in the first place. The solutions in the GoF patterns book are mostly aimed at a C++ implementation. Even if they apply to your firmware — some of them do! — you will probably need to translate them into your own implementation language (probably C). This is part of using patterns anyway: the solutions recommended are general ones. You need to adapt them to your particular needs.

In firmware, our implementations are constrained to be simple and simplified, because we don’t have the resources (RAM, processor power, etc.) to support the techniques and languages that many of our software peers enjoy. This simplification may lead some to think that our firmware solutions have over-simplified the pattern. Are we really using software patterns? Yes, I think we are, if our applications use solutions that are informed by the original patterns.

Here is an example from my own experience. If you think I didn’t really use patterns properly, I’m not going to argue with you. The important thing is that I found the patterns helpful in coming to a successful and workable solution to my problem. Investigate patterns for yourself, draw from them anything you find valuable, and use it.

On an OO project using C++, I designed a message handler. Our target hardware connected to its host via a serial link. When a message from the master arrived, it had to be processed. I combined the Command and Factory patterns to come up with a solution.

When a message arrived, and had been validated, its payload was passed to a Factory method that created a command instance whose type matched the message (command) just received. The Factory returned the new instance using a pointer to the abstract command type. The application then called the Execute() method of the returned instance (which processed the message as required) and then deleted it. An elegant, if simple, use of patterns to solve the problem.

Later in my career, I revisited this solution in C. We used a very simple approximation to OO that we called ‘classes in C’. Many of the nuts and bolts that an OO language would have hidden are explicitly coded in C. And so my solution comprised a message-handler class with an API method — i.e. a public class method — to which the message payload was passed. This method extracted the command type from the payload, and used a simple jump table to execute the appropriate method, and process the message. [It is worth noting here that the message-processing code is exactly the same in the C and C++ examples; it’s how you get to it that’s a little different.]

In both cases, a simple and extensible technique is used to connect a particular message with the code that will process that message. In both cases, the Factory and Command patterns inform the solution. I would say that I used patterns in my firmware. But, as I have said, I won’t argue with you if you disagree. The important thing is the solution, which stands for itself.

In a comment to a previous blog post, Matt Chernosky asked for more details on using patterns in firmware. This is your answer, Matt. Is it what you hoped for?

This is the fifth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … software patterns?

Why does my firmware need … a Quality Process?

This post is a little different from the others in this series. Not every firmware designer will benefit from using a Quality Process. But I’m getting ahead of myself. The first thing we need is some background.

Quality Processes lost their good name in the bad olde days B.A. (Before Agile), and deservedly so. These were the days when we pretended to use the waterfall design methodology. Everyone knew that the emperor was naked, but developers and managers alike were buried under the lies. We all knew waterfall didn’t work, so we did what we do now (what actually does work), and we pretended to follow waterfall. We produced some documents whose only purpose was to satisfy our Quality Processes, and we did some other useless stuff too.

The fault here belongs with the layer of lies we constructed to bolster our pretence that waterfall described how we really worked. But Quality got stuck with the blame.

A Quality Process supports, nurtures and encourages the creation of quality firmware. [I won’t get stuck, as Phaedrus did, with trying to define “quality”. We all know it when we see it.] If it doesn’t, it isn’t a Quality process, by definition. Those olde processes, and the useless documents they made us write, didn’t help the quality of our firmware.

Another problem with some real world Quality Processes is that they were written by people who knew little of what quality firmware is, and how its creation could or should be encouraged. I have even encountered a Quality Process that deliberately included useless requirements, so that the author could discipline staff who didn’t adhere to them. Thankfully, this was not in the software business.

There is another problem too: including unnecessary details in necessary constraints. In one company I worked for, our process required that a particular text editor be used to produce a simple ASCII-text document. I pointed out to our Quality Champion that the requirement concerned the document format, and could reasonably be created with any text editor. He couldn’t see what my problem was, and chose not to consider the matter further. I made no further attempts at Quality Improvement while I worked with that team.

A good Quality Process places on developers only those constraints that are really needed if our firmware is to be of good quality. Most professional firmware designers could be helped in their work by such a process. People are the problem, not Quality. Good Quality Processes can be created, and your team deserves one.

Our Quality Process mandated the use of a coding standard, but did not specify the standard itself. Control of the coding standard (and its contents) was deliberately left under the team’s control; our process only required that the team should use one.

If your development is governed by ISO9000, then you must have a Quality Process. But you are allowed to have a good one! All ISO9000 requires is that you document how you create your firmware, do it, and record that you did. And that’s nearly it.

Your Quality Process also needs to show that your development is under control, and how it is kept so. For example, build release is not the time to discover a fundamental flaw in your design. We need to monitor how things are going while development is in progress. In our process, we kept the checks to a minimum. For each module (big or small), the designer had to present a design document for review. This document had two purposes: to verify that the design is good enough for development to continue, and to explain to a new member of the team what your module does, and how it does it. The only other check was a review of the module immediately before sign-off. We found these checks to be quite enough.

A final remark about Quality Processes: they should be written on the basis that the team is composed of professional and trustworthy designers, not reckless miscreants who must be kept in order by threats.

So, does our firmware need a Quality Process? This leads us to a Clintonesque ‘define-sex’ moment, as we wonder exactly what is meant by need. I think it’s fair to say that most of us could benefit from a good Quality Process, even those of us who aren’t required by our employers to have one. But a bad Quality Process is death to quality firmware. If you are subject to a bad process, get it changed, or move on, that’s my advice.

If you’ve got here, and now you’re wondering why I haven’t told you how to write a good Quality Process, you need help. And I mean that in the nicest possible way. To write a Quality Process, you must know what quality firmware is, and how its creation could or should be encouraged. If you don’t, seek out someone who does, and persuade them to help you. The end result really will help your development.

This is the fourth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you have to add to the discussion! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … a Quality Process?

Why does my firmware need … an RTOS?

Once upon a time, there was a firmware development team. For many years, the team had been working on an RTOS spell, and now, finally, it was complete. They had tested it in a variety of their arcane firmware rituals, and it had worked well. The team were well pleased, and proud of what they had wrought.

From their caverns, deep in the darkest recesses of the world — where firmware designers must dwell, for they are shunned by all other mortals — they sent word to other teams in their company, offering them the use of their spell. Instead of being showered by their peers with praise and gratitude, the only response they received was as terse as it was unexpected:

We have never seen the need for an RTOS.

If this really was a fairy tale, there would be no reason for this post. But every word of it is true. The response came from the leader of an established team, with quite a few products and projects to their name. And it wasn’t that long ago. They had never seen the need for an RTOS…. <despair> <sympathy> <disbelief>

The title of this post asks why our firmware needs an RTOS, but the truth is that it already has one. If you think yours doesn’t, that’s because you haven’t split it out from your application code. [Shame on you!] Your application already has hardware initialisation code, peripheral device drivers, tasks and timers. That’s your RTOS. That’s what an RTOS is, and what it does.

You don’t use tasks? Then what is that loop at the end of your main(), that executes a number of methods in sequence? It’s a simple task scheduler, and the methods are tasks.

You already have an RTOS. In accordance with DRY, split that code out, and use the one tried and tested copy in all your future applications. Using an RTOS is simple code re-use. I know of no argument against that, do you?

Having limited resources on your target is no barrier to using an RTOS. We used ours without problems on targets with as little as 256 bytes of RAM. Call it an Application Support Layer if you don’t like the usual name, but your firmware needs an RTOS.

If a commercial RTOS meets your needs, use one. If not, and your needs are simple, write your own. Most of your RTOS code is peripheral drivers anyway, and your build-tool or micro-controller vendor will usually supply them free. We wrote our own, and never regretted it.

This is the third in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you think. Thanks for dropping by!


[Edited to remove the half-line that crept in at the beginning. Sorry about that.]


Why does my firmware need … an RTOS?

Why does my firmware need … unit-tests?

It’s nice to have a suite of tests that confirms our public/API methods work as they should, especially if we’re trying to decide if the current build is ready for acceptance testing. But that only makes unit-testing ‘good to have’. We need unit-tests because they enable refactoring. The argument goes like this:

  • Software rot is terminal. A risky and expensive rewrite is unavoidable.
  • The only thing (I know of) that can prevent software rot is refactoring.
  • Refactoring without a way to check we haven’t introduced new bugs into our production code is reckless and irresponsible!
  • Unit-tests check that these changes haven’t introduced new bugs into our code.
  • Therefore unit-tests are essential to keep our code extensible and maintainable.

This is why our firmware needs unit-tests. But unit-testing can deliver more, if we also adopt test-driven design (TDD). Test-after (writing the code) is tedious in the extreme. Therefore, being human, we will — in practice and in general — avoid writing them if we can. The test-first approach intersperses writing tests and production code, making it much less tedious, and encouraging us to write more and better tests. But that isn’t the main benefit of TDD.

For experienced practitioners, the mnemonic “red, green, refactor” is sufficient. But to see the benefits of TDD, we need to consider it in more detail:

  • Add a test for a feature to be added to the method-under-test.
  • Run all tests and confirm the new one fails.
  • Write the simplest code that will pass the new test.
  • Run all tests and confirm they pass.
  • Refactor code.
  • Run all tests and confirm they pass.
  • Repeat (if there are still features to be added).
  • The method-under-test is complete.

Before we write a test, we must know what we want to test. TDD makes us focus on the requirements before we write the code. XP/Lean/Agile wisely warns us away from too much up-front design. Here, TDD helps us check we have done just enough.

Odd as it may seem, there is no obvious way to know exactly when you have finished coding a method. Often, we just stop when we feel we’ve done enough. TDD tells us when we’ve finished: when all required features are present, tested and working.

The benefits of refactoring cannot be over-emphasised, in my opinion. It’s not just the (rather negative) avoidance of software rot, it’s the greater coherence our designs have. There are lots of ‘code smells’ that prompt us to refactor, but the simplest is: if you spot a bolt-on in your code, refactor until it belongs. This is particularly relevant during TDD, where we first just code a solution, then refactor it into the local design structure.

Finally, we come to the most obvious benefit of all: TDD really does reduce bugs. Microsoft and IBM did some research, and found bug-counts reduced by 60–90%.

This is the second in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you think. Thanks for dropping by!

Why does my firmware need … unit-tests?

Why does my firmware need … design?

It doesn’t! There is no part of the process of creating an executable to meet your customer’s needs that requires design. It’s you that needs it, because you’re human. Humans are not good at dealing with complexity. Without mnemonic tricks, we can only hold three or four things in our minds at one time, and so on. Over the millennia, we have developed techniques to help us deal with complexity. Collectively, we call them ‘design’.

It is perfectly possible to create firmware executables without applying design; we can just sit down and ‘do it’. XP first emerged as a reaction to the design-heavy processes and methodologies prevalent at the time. In the way of humans, some practitioners pounced on XP as a reason for doing no design at all. This is a misunderstanding. We need to return to the middle path, whereby just enough design, and no more, is the preferred approach.

Design is the name we give to the toolset we use to produce our firmware. A wood-cutter could fell a tree using only teeth and fingernails, but an axe offers considerable advantages! Design helps us to produce a structured response to our customers’ needs, and that structure helps us humans to manage and understand our own designs.

Abstraction is an important member of our design toolset. It involves stripping away detail so that we can more easily see the underlying structure. By creating a layered, abstracted, design, we can divide and conquer, somewhat akin to the application of reductionism in science. We divide each larger problem into a collection of smaller, simpler problems that are easier for us to deal with. We do this enough times that the small problems we end up with can be understood and solved easily.

So this is why our firmware needs design. And, just as our programs should be as simple as they can be, but no simpler, they need just enough design, but no more.

This is the first in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you have to add to the discussion!

Why does my firmware need … design?