Why does my firmware need … humans?

Unlike the other posts in this series, human influence on our firmware is not central to firmware development. However, it is something we all should look at, straight in the eyes, at least once. I’m going to assume that this is that once.

Snap question: Why does my firmware need humans?

Barely-considered response: We need humans as users and customers, because without them there could be no product and no firmware, and because they are the original source of the requirements our firmware must meet.

Just as science mostly ignores humans, or reduces them to impartial observers (nullifying any possible active influence they may exert), we tend to consider human firmware developers as logical and rational beings. Humans can, of course, act rationally and logically, but only sometimes. At other times we are anything but. This is our nature; let’s not deny it, or cry about it, but let’s admit it, and allow for it.

Our users and customers are outside our businesses, but humans within our businesses occupy quite a few roles in our firmware development ecosystem. They include the firmware design team, maybe a software and/or hardware team, your immediate (technical) manager (team leader?), a Product Manager, a Sales Manager, a Project Manager, maybe a QA testing team, and so on. All of these roles should and do exert a direct influence on your firmware development. Some humans may play more than one role, depending on how your company is organised. Their collective influence can be summed up in one word: uncertainty.

Some uncertainties emerge from the roles, for example a mid-project requirements change. We don’t know if they will happen, although past experience indicates they often do. If they do happen, we don’t know when in the project timeline, and we don’t know their exact nature, so we don’t know how significantly they will affect the current state of our firmware design. Uncertainty.

Other uncertainties emerge from the humans playing these roles, and they are all errors of one sort or another. The most obvious example is bugs in your code. As before, we don’t know if or when they will happen, when and where they will be discovered, and we don’t know how badly they will affect the operation of our firmware. Uncertainty again.

None of this should be a surprise to you, but I don’t think we account for these uncertainties as we should, in our work, and in our development processes. We can and do exhibit logical rational behaviour, but not all the time. We should account for this, to the extent it’s practical and useful.

We have already made progress on this topic. We used to freeze requirements during the lifetime of a project. Now we understand that mid-project requirement changes are a commercial reality, and we allow for them (however reluctantly). But there is more to human influence (on our firmware) than requirements changes.

The roles I listed control the availability of equipment, the developers available to the current project, access to tools and training, support (or the lack of it) for professional design practices, and so on. Can your firmware development process usefully allow for any/all of these?

Edited to add> Here’s an uncertainty that I always overlook. [Shame on me. 😦 ] We ask our firmware designers to solve problems that have not been solved before (otherwise we’d use the existing solutions). To do this, they sometimes have to think of things that haven’t been tried before, and sometimes this leads them to ‘break the rules’, or maybe make new ones. We ask our designers to use imagination and creativity, the outcome of which is unpredictable. From the perspective of our firmware, this is another uncertainty. And it’s meant to be there.

The uncertainties that stem from humans, rather than the roles they play, are basically errors of some sort. But it is a mistake to think that coding bugs are all there is to it. Any part/aspect of a design that is incomplete or incorrect is an error. But there’s much more to it than even this.

Of all the humans who exert influence on our firmware development, we should ask: are they competent to enact their roles? [We are not excluded from this!] Do we get the resources we need to complete our projects? Do we use them as well as we should?  Can your firmware development process usefully allow for any/all of these?

With all these humans involved, we would be remiss to ignore communication. An incorrect or incomplete communication can wreak havoc during firmware development. In the worst case (or a pretty bad one), the information you need to form the requirements your firmware must meet might reach you by the following route: users -> customer -> sales representative -> Sales Manager -> Product Manager -> your line manager -> you. The words “Chinese” and “whispers” should be dominating our thoughts right now. 🙂 Add to this that many of these humans have their own agendas, prompting them to ‘interpret’ the information as it passes by, for the benefit of all (let’s be kind).   Can your firmware development process usefully allow for any/all of these?

In the case of human error, is it enough simply to adopt practices to minimise them, or should we also allow for them, and maybe introduce ways of tolerating them (resilience)?

You won’t be considering all this stuff again (probably), so think: are there any other examples I haven’t written down here? How do they affect your firmware development?

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … humans?

Why does my firmware need … documentation?

Documentation does not exist in isolation. It is justified in the context of your development ecosystem. Since I don’t know how you work, I will describe how we used documentation, and why.

Our Quality Process was ISO9000-compliant. Among other things, we needed to show that our development was controlled. Our Project team also needed us to monitor progress. It is too risky to wait until a development is complete to discover problems. But this control was not something imposed on us, it was a sensible practice we all accepted.

For every module, of whatever size, we followed a two-stage process: plan it, then do it. [There was some to-ing and fro-ing between the two, but that’s not what we’re focussing on here.] There was only room for two control points, a review of the plan, and a review of the final output, so that’s how we worked. Our designers produced a design document, describing their plan, and the team reviewed it, to confirm that development should continue.

Among other things, our review of the plan confirmed that it left space for coding, which is also design. Too much design up-front is as bad as too little. Reviewing the design document allowed us to make these checks.

This was why we found our firmware needed documentation. I believe that similar reasoning applies to your design and development process too, but only you can be the judge of that. At least consider the matter carefully: does your firmware need documentation? If so, what, and how much?

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … documentation?

Why does my firmware need … a Quality Process? (2)

In my last post, I said I couldn’t write your Quality Process for you, and I can’t. That’s because your process needs to be personalised (or ‘teamalised’, even though it isn’t a proper word). Only you and your team know how you work, which is the core content of your QP document. But there is guidance, and there are hints, that I can offer.

Never ever include unnecessary detail. [This applies to all technical documents.] This guideline benefits you at least as much as it benefits anyone or anything else. There are a surprising number of justifications for this. Here are a few of them.

  • If you commit yourself to producing a design that is green (I know; just trying to keep my analogies simple), then find during coding (which is also design) that it needs to be blue, you have to go back and change your document too. Just say the design will be coloured to optimise reflectivity (or whatever), or don’t mention colour at all, unless it’s a significant attribute of your design.
  • If you specify too much at any stage or abstract level, you risk doing too much up front, (unnecessarily) constraining the design that still remains to be done.
  • Unnecessary detail takes attention away from the significant things you have to say.
  • Unnecessary detail often breaks DRY, which is something we never do. 😉

Write your QP for trusted firmware designers, not for miscreants that must be kept in order. We are all professionals; mutual respect is essential.

Never add anything to your QP that isn’t true. It will be no good to you if you do. But when you write it down for the first time, you may see that the way you work is not how you think you should work. Good! Agree the right way to work with all involved parties, change over to the new practice, and then describe it in your QP. Your QP has already benefitted your work.

Every firmware project is subject to requirements that change (or are added) during its lifetime. The development process you describe should describe how you deal with this. Agile techniques offer possible answers; I’m sure there are others.

If you always spend considerable time debugging, include that in your QP description. [Then, perhaps, you might want to give some thought about how you can spend more time on the good stuff, and less on debugging. 🙂 Dare I suggest TDD?] If you carry out, or assist in, acceptance testing, include that too. Don’t miss out any contribution you make to the development process.

When you have finished, you will have a checklist, useful to the newbie and the experienced developer alike. It will be especially helpful if you are unlucky enough to work without the support of a team. It will help your managers too.

Hope this helps?

This is an addendum to the fourth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … a Quality Process? (2)

Why does my firmware need … software patterns?

Our firmware needs software patterns as much as any software does. Which is to say we don’t need them, but they are a valuable tool. A pattern is written by a designer experienced enough to have come across many specific problems, and recognise them as instances of one general problem. Then it offers a general solution that has worked well, explaining why it might suit your specific problem, and (more importantly?) why it might not. A pattern is a valuable capsule of design information.

As a firmware designer, you will find that most (not all!) published patterns describe problems that you do not, and will not, encounter. [This applies to all users of patterns.] Inasmuch as anyone is to blame, this is our fault. It takes seasoned firmware designers to recognise and describe the problems that occur commonly in our work. If we want more firmware patterns, we need to write them…. [There are a couple of books — here and here — that look promising; I haven’t read them. If you have, please let us know what you thought of them. Thanks.]

The solutions recommended in patterns are dependent on various things. The implementation language is one of them. A pattern that is useful for one language might be unnecessary in another, whose native features automatically solve the problem, or prevent it from occurring in the first place. The solutions in the GoF patterns book are mostly aimed at a C++ implementation. Even if they apply to your firmware — some of them do! — you will probably need to translate them into your own implementation language (probably C). This is part of using patterns anyway: the solutions recommended are general ones. You need to adapt them to your particular needs.

In firmware, our implementations are constrained to be simple and simplified, because we don’t have the resources (RAM, processor power, etc.) to support the techniques and languages that many of our software peers enjoy. This simplification may lead some to think that our firmware solutions have over-simplified the pattern. Are we really using software patterns? Yes, I think we are, if our applications use solutions that are informed by the original patterns.

Here is an example from my own experience. If you think I didn’t really use patterns properly, I’m not going to argue with you. The important thing is that I found the patterns helpful in coming to a successful and workable solution to my problem. Investigate patterns for yourself, draw from them anything you find valuable, and use it.

On an OO project using C++, I designed a message handler. Our target hardware connected to its host via a serial link. When a message from the master arrived, it had to be processed. I combined the Command and Factory patterns to come up with a solution.

When a message arrived, and had been validated, its payload was passed to a Factory method that created a command instance whose type matched the message (command) just received. The Factory returned the new instance using a pointer to the abstract command type. The application then called the Execute() method of the returned instance (which processed the message as required) and then deleted it. An elegant, if simple, use of patterns to solve the problem.

Later in my career, I revisited this solution in C. We used a very simple approximation to OO that we called ‘classes in C’. Many of the nuts and bolts that an OO language would have hidden are explicitly coded in C. And so my solution comprised a message-handler class with an API method — i.e. a public class method — to which the message payload was passed. This method extracted the command type from the payload, and used a simple jump table to execute the appropriate method, and process the message. [It is worth noting here that the message-processing code is exactly the same in the C and C++ examples; it’s how you get to it that’s a little different.]

In both cases, a simple and extensible technique is used to connect a particular message with the code that will process that message. In both cases, the Factory and Command patterns inform the solution. I would say that I used patterns in my firmware. But, as I have said, I won’t argue with you if you disagree. The important thing is the solution, which stands for itself.

In a comment to a previous blog post, Matt Chernosky asked for more details on using patterns in firmware. This is your answer, Matt. Is it what you hoped for?

This is the fifth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … software patterns?

Why does my firmware need … a Quality Process?

This post is a little different from the others in this series. Not every firmware designer will benefit from using a Quality Process. But I’m getting ahead of myself. The first thing we need is some background.

Quality Processes lost their good name in the bad olde days B.A. (Before Agile), and deservedly so. These were the days when we pretended to use the waterfall design methodology. Everyone knew that the emperor was naked, but developers and managers alike were buried under the lies. We all knew waterfall didn’t work, so we did what we do now (what actually does work), and we pretended to follow waterfall. We produced some documents whose only purpose was to satisfy our Quality Processes, and we did some other useless stuff too.

The fault here belongs with the layer of lies we constructed to bolster our pretence that waterfall described how we really worked. But Quality got stuck with the blame.

A Quality Process supports, nurtures and encourages the creation of quality firmware. [I won’t get stuck, as Phaedrus did, with trying to define “quality”. We all know it when we see it.] If it doesn’t, it isn’t a Quality process, by definition. Those olde processes, and the useless documents they made us write, didn’t help the quality of our firmware.

Another problem with some real world Quality Processes is that they were written by people who knew little of what quality firmware is, and how its creation could or should be encouraged. I have even encountered a Quality Process that deliberately included useless requirements, so that the author could discipline staff who didn’t adhere to them. Thankfully, this was not in the software business.

There is another problem too: including unnecessary details in necessary constraints. In one company I worked for, our process required that a particular text editor be used to produce a simple ASCII-text document. I pointed out to our Quality Champion that the requirement concerned the document format, and could reasonably be created with any text editor. He couldn’t see what my problem was, and chose not to consider the matter further. I made no further attempts at Quality Improvement while I worked with that team.

A good Quality Process places on developers only those constraints that are really needed if our firmware is to be of good quality. Most professional firmware designers could be helped in their work by such a process. People are the problem, not Quality. Good Quality Processes can be created, and your team deserves one.

Our Quality Process mandated the use of a coding standard, but did not specify the standard itself. Control of the coding standard (and its contents) was deliberately left under the team’s control; our process only required that the team should use one.

If your development is governed by ISO9000, then you must have a Quality Process. But you are allowed to have a good one! All ISO9000 requires is that you document how you create your firmware, do it, and record that you did. And that’s nearly it.

Your Quality Process also needs to show that your development is under control, and how it is kept so. For example, build release is not the time to discover a fundamental flaw in your design. We need to monitor how things are going while development is in progress. In our process, we kept the checks to a minimum. For each module (big or small), the designer had to present a design document for review. This document had two purposes: to verify that the design is good enough for development to continue, and to explain to a new member of the team what your module does, and how it does it. The only other check was a review of the module immediately before sign-off. We found these checks to be quite enough.

A final remark about Quality Processes: they should be written on the basis that the team is composed of professional and trustworthy designers, not reckless miscreants who must be kept in order by threats.

So, does our firmware need a Quality Process? This leads us to a Clintonesque ‘define-sex’ moment, as we wonder exactly what is meant by need. I think it’s fair to say that most of us could benefit from a good Quality Process, even those of us who aren’t required by our employers to have one. But a bad Quality Process is death to quality firmware. If you are subject to a bad process, get it changed, or move on, that’s my advice.

If you’ve got here, and now you’re wondering why I haven’t told you how to write a good Quality Process, you need help. And I mean that in the nicest possible way. To write a Quality Process, you must know what quality firmware is, and how its creation could or should be encouraged. If you don’t, seek out someone who does, and persuade them to help you. The end result really will help your development.

This is the fourth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you have to add to the discussion! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … a Quality Process?

Why does my firmware need … an RTOS?

Once upon a time, there was a firmware development team. For many years, the team had been working on an RTOS spell, and now, finally, it was complete. They had tested it in a variety of their arcane firmware rituals, and it had worked well. The team were well pleased, and proud of what they had wrought.

From their caverns, deep in the darkest recesses of the world — where firmware designers must dwell, for they are shunned by all other mortals — they sent word to other teams in their company, offering them the use of their spell. Instead of being showered by their peers with praise and gratitude, the only response they received was as terse as it was unexpected:

We have never seen the need for an RTOS.

If this really was a fairy tale, there would be no reason for this post. But every word of it is true. The response came from the leader of an established team, with quite a few products and projects to their name. And it wasn’t that long ago. They had never seen the need for an RTOS…. <despair> <sympathy> <disbelief>

The title of this post asks why our firmware needs an RTOS, but the truth is that it already has one. If you think yours doesn’t, that’s because you haven’t split it out from your application code. [Shame on you!] Your application already has hardware initialisation code, peripheral device drivers, tasks and timers. That’s your RTOS. That’s what an RTOS is, and what it does.

You don’t use tasks? Then what is that loop at the end of your main(), that executes a number of methods in sequence? It’s a simple task scheduler, and the methods are tasks.

You already have an RTOS. In accordance with DRY, split that code out, and use the one tried and tested copy in all your future applications. Using an RTOS is simple code re-use. I know of no argument against that, do you?

Having limited resources on your target is no barrier to using an RTOS. We used ours without problems on targets with as little as 256 bytes of RAM. Call it an Application Support Layer if you don’t like the usual name, but your firmware needs an RTOS.

If a commercial RTOS meets your needs, use one. If not, and your needs are simple, write your own. Most of your RTOS code is peripheral drivers anyway, and your build-tool or micro-controller vendor will usually supply them free. We wrote our own, and never regretted it.

This is the third in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you think. Thanks for dropping by!


[Edited to remove the half-line that crept in at the beginning. Sorry about that.]


Why does my firmware need … an RTOS?

Why does my firmware need … unit-tests?

It’s nice to have a suite of tests that confirms our public/API methods work as they should, especially if we’re trying to decide if the current build is ready for acceptance testing. But that only makes unit-testing ‘good to have’. We need unit-tests because they enable refactoring. The argument goes like this:

  • Software rot is terminal. A risky and expensive rewrite is unavoidable.
  • The only thing (I know of) that can prevent software rot is refactoring.
  • Refactoring without a way to check we haven’t introduced new bugs into our production code is reckless and irresponsible!
  • Unit-tests check that these changes haven’t introduced new bugs into our code.
  • Therefore unit-tests are essential to keep our code extensible and maintainable.

This is why our firmware needs unit-tests. But unit-testing can deliver more, if we also adopt test-driven design (TDD). Test-after (writing the code) is tedious in the extreme. Therefore, being human, we will — in practice and in general — avoid writing them if we can. The test-first approach intersperses writing tests and production code, making it much less tedious, and encouraging us to write more and better tests. But that isn’t the main benefit of TDD.

For experienced practitioners, the mnemonic “red, green, refactor” is sufficient. But to see the benefits of TDD, we need to consider it in more detail:

  • Add a test for a feature to be added to the method-under-test.
  • Run all tests and confirm the new one fails.
  • Write the simplest code that will pass the new test.
  • Run all tests and confirm they pass.
  • Refactor code.
  • Run all tests and confirm they pass.
  • Repeat (if there are still features to be added).
  • The method-under-test is complete.

Before we write a test, we must know what we want to test. TDD makes us focus on the requirements before we write the code. XP/Lean/Agile wisely warns us away from too much up-front design. Here, TDD helps us check we have done just enough.

Odd as it may seem, there is no obvious way to know exactly when you have finished coding a method. Often, we just stop when we feel we’ve done enough. TDD tells us when we’ve finished: when all required features are present, tested and working.

The benefits of refactoring cannot be over-emphasised, in my opinion. It’s not just the (rather negative) avoidance of software rot, it’s the greater coherence our designs have. There are lots of ‘code smells’ that prompt us to refactor, but the simplest is: if you spot a bolt-on in your code, refactor until it belongs. This is particularly relevant during TDD, where we first just code a solution, then refactor it into the local design structure.

Finally, we come to the most obvious benefit of all: TDD really does reduce bugs. Microsoft and IBM did some research, and found bug-counts reduced by 60–90%.

This is the second in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Let us know what you think. Thanks for dropping by!

Why does my firmware need … unit-tests?