Why does my firmware need … honesty?

There are some things that have been believed for so long, and so universally, we don’t think to question them. Mostly, this makes sense: we believe them because they’re right. But it isn’t true in every case.

The waterfall method of design is an infamous example. It took a carefully-applied dose of doublethink to do what actually worked, while still believing that waterfall was correct. We weren’t deliberately lying to ourselves or each other; we didn’t (consciously) realise what we were doing. And this is the sort of (dis)honesty I’m referring to. Intentional deception is very rare in firmware design, in my experience.

I think I may have spotted something of this kind. Let’s see if you agree. 🙂 Many commentators publish articles and books containing good and wise advice. [No irony intended: their advice is good and wise, as far as I can tell.] But they often seem to assume that we (designers) are all good (enough) at what we do, and that we want to get better at it. There are many working developers who do not meet these criteria. This matters because the advice often depends for its goodness and wisdom on its recipients (us) being both able and willing to take its recommendations on board.

Ego-less code reviews, for example, are often recommended, and with good reason. But some of us can only manage this some of the time, and some of us can’t do it at all. We’re human! We exhibit human failings. So ego-assassination code reviews are more common than we would prefer, with all the negative consequences we might expect. [Better not to review at all than this?]

I’m not attacking humans. What would be the point? 😉 But I do think the design practices we create and recommend should be suitable for real designers, not just for Vulcans or androids. I think there’s a point in that, and I think it may be one of those things we’ve accidentally ignored for too long. What do you think? Do I have a point? Are there other things we ignore, that we should look at more carefully and more critically? How about leaving a comment? 😉

“Who cares, wins.”

This blog post is the latest in a series that is longer than I expected it to be.:) Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … honesty?

Why does my firmware need … state?

State — that’s static variables in firmware-speak — is considered harmful by hipster designers. But they are as different from us as two software specialisations can be. They live on the Ethereal Plane. [Why do you think it’s called “Ethernet”? 🙂] They belong to different schools of magick from us, they cast different spells, with different intent, in their world of nets and webs. State introduces problems for them.

The arguments hipsters have put together to vilify state have merit, but many of their objections describe problems with state in the context of their Ethereal Plane. They do not affect us or our firmware. What remains after we discard these context-specific objections is that state represents a hidden parameter passed to every method when it’s called. No programming technique is without its downside, and that is a downside of state.

State is unavoidable in firmware. Our applications contain tasks that are continually executed. In between task invocations, the state of each task is stored. When that task is executed again, its state tells it where it has got to, and (therefore) what to do next. Unless we make radical changes to the way we design — and this is not impossible… — we need state.

Another objection to state is that it makes code more difficult to test. It does, but so does any addition to existing code. Instead of just testing a method once, we have to run each test in every possible state. This is not a reason for us to give up on state, but it is a reason to minimise it. An application with too much state becomes difficult to understand and, eventually, impossible to extend or maintain. Clearly, some caution is appropriate in our use of state. But caution is appropriate to all of our coding, whatever we’re doing. It doesn’t just apply to state.

Hipsters found problems, and traced them to state. At this point I think they adopted the belief that state is bad. Then they searched for justifications for their beliefs, and they found some. That some or all of their arguments are valid doesn’t change what they have done, and are doing. They seek to optimise their working practices for their world, and the work they do, which is eminently sensible. And for them, the bad points of state outweigh the good ones. For us, that is not necessarily the case, and we should resist being carried along with the tide, just because that’s what hipsters are doing.

State is not harmful, but it should be used with care.

“Who cares, wins.”

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … state?

Why does my firmware need … architecture?

Every design has its architecture, but if your design is small enough, you may not need to consider it separately. I will qualify this heresy after I’ve defined how I use a couple of very important words.

Firmware — The operational program that runs on an embedded processor with significant resource constraints. Although fast 32-bit micro-controllers are now commonplace, 8 kBytes of RAM is quite normal, and 64 kBytes is undreamed-of luxury. Supported (and supportable) languages are assembler, C and C++, although the latter is rarely used. Resource limitations mean that firmware cannot support WinCE, embedded Linux or Android.

Architecture — Like “design”, “architecture” is difficult to define. It is a superficial overview of your design, but it extends down to the smallest details. Grady Booch said “All architecture is design, but not all design is architecture”. Architecture is the collected features — the significant ones — that define your design. But it is also the collected details that enable these features to be reified. Consider your design as a flower, and each significant feature as a petal. Your architecture must describe each petal well enough for it to be designed and built, but it must also describe how the features (petals) blend together to make a flower. At the detail level, even petal-attachment protocols may be needed. 😉

So why does my firmware need architecture? Like RTOSs (link), your firmware has architecture anyway, so “need” doesn’t really come into it. It has design too (link), which incorporates your architecture. In software, it is normal and desirable – if not mandatory — to consider the application’s architecture separately. This should sometimes be the case for firmware too. But, for the smallest firmware, the design is compact enough that splitting it into architecture and ‘the rest’ is one simplification beyond necessity. The whole is simple enough that it is acceptable just to consider its design (where that design silently embraces and includes the architecture).

You can tell if you should consider your application’s architecture separately by answering this simple question: Will my design be clearer and easier to understand if I split it into architecture and the rest? Just enough design applies to architecture too.

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … architecture?

Why does my firmware need … humans?

Unlike the other posts in this series, human influence on our firmware is not central to firmware development. However, it is something we all should look at, straight in the eyes, at least once. I’m going to assume that this is that once.

Snap question: Why does my firmware need humans?

Barely-considered response: We need humans as users and customers, because without them there could be no product and no firmware, and because they are the original source of the requirements our firmware must meet.

Just as science mostly ignores humans, or reduces them to impartial observers (nullifying any possible active influence they may exert), we tend to consider human firmware developers as logical and rational beings. Humans can, of course, act rationally and logically, but only sometimes. At other times we are anything but. This is our nature; let’s not deny it, or cry about it, but let’s admit it, and allow for it.

Our users and customers are outside our businesses, but humans within our businesses occupy quite a few roles in our firmware development ecosystem. They include the firmware design team, maybe a software and/or hardware team, your immediate (technical) manager (team leader?), a Product Manager, a Sales Manager, a Project Manager, maybe a QA testing team, and so on. All of these roles should and do exert a direct influence on your firmware development. Some humans may play more than one role, depending on how your company is organised. Their collective influence can be summed up in one word: uncertainty.

Some uncertainties emerge from the roles, for example a mid-project requirements change. We don’t know if they will happen, although past experience indicates they often do. If they do happen, we don’t know when in the project timeline, and we don’t know their exact nature, so we don’t know how significantly they will affect the current state of our firmware design. Uncertainty.

Other uncertainties emerge from the humans playing these roles, and they are all errors of one sort or another. The most obvious example is bugs in your code. As before, we don’t know if or when they will happen, when and where they will be discovered, and we don’t know how badly they will affect the operation of our firmware. Uncertainty again.

None of this should be a surprise to you, but I don’t think we account for these uncertainties as we should, in our work, and in our development processes. We can and do exhibit logical rational behaviour, but not all the time. We should account for this, to the extent it’s practical and useful.

We have already made progress on this topic. We used to freeze requirements during the lifetime of a project. Now we understand that mid-project requirement changes are a commercial reality, and we allow for them (however reluctantly). But there is more to human influence (on our firmware) than requirements changes.

The roles I listed control the availability of equipment, the developers available to the current project, access to tools and training, support (or the lack of it) for professional design practices, and so on. Can your firmware development process usefully allow for any/all of these?

Edited to add> Here’s an uncertainty that I always overlook. [Shame on me. 😦 ] We ask our firmware designers to solve problems that have not been solved before (otherwise we’d use the existing solutions). To do this, they sometimes have to think of things that haven’t been tried before, and sometimes this leads them to ‘break the rules’, or maybe make new ones. We ask our designers to use imagination and creativity, the outcome of which is unpredictable. From the perspective of our firmware, this is another uncertainty. And it’s meant to be there.

The uncertainties that stem from humans, rather than the roles they play, are basically errors of some sort. But it is a mistake to think that coding bugs are all there is to it. Any part/aspect of a design that is incomplete or incorrect is an error. But there’s much more to it than even this.

Of all the humans who exert influence on our firmware development, we should ask: are they competent to enact their roles? [We are not excluded from this!] Do we get the resources we need to complete our projects? Do we use them as well as we should?  Can your firmware development process usefully allow for any/all of these?

With all these humans involved, we would be remiss to ignore communication. An incorrect or incomplete communication can wreak havoc during firmware development. In the worst case (or a pretty bad one), the information you need to form the requirements your firmware must meet might reach you by the following route: users -> customer -> sales representative -> Sales Manager -> Product Manager -> your line manager -> you. The words “Chinese” and “whispers” should be dominating our thoughts right now. 🙂 Add to this that many of these humans have their own agendas, prompting them to ‘interpret’ the information as it passes by, for the benefit of all (let’s be kind).   Can your firmware development process usefully allow for any/all of these?

In the case of human error, is it enough simply to adopt practices to minimise them, or should we also allow for them, and maybe introduce ways of tolerating them (resilience)?

You won’t be considering all this stuff again (probably), so think: are there any other examples I haven’t written down here? How do they affect your firmware development?

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … humans?

Why does my firmware need … documentation?

Documentation does not exist in isolation. It is justified in the context of your development ecosystem. Since I don’t know how you work, I will describe how we used documentation, and why.

Our Quality Process was ISO9000-compliant. Among other things, we needed to show that our development was controlled. Our Project team also needed us to monitor progress. It is too risky to wait until a development is complete to discover problems. But this control was not something imposed on us, it was a sensible practice we all accepted.

For every module, of whatever size, we followed a two-stage process: plan it, then do it. [There was some to-ing and fro-ing between the two, but that’s not what we’re focussing on here.] There was only room for two control points, a review of the plan, and a review of the final output, so that’s how we worked. Our designers produced a design document, describing their plan, and the team reviewed it, to confirm that development should continue.

Among other things, our review of the plan confirmed that it left space for coding, which is also design. Too much design up-front is as bad as too little. Reviewing the design document allowed us to make these checks.

This was why we found our firmware needed documentation. I believe that similar reasoning applies to your design and development process too, but only you can be the judge of that. At least consider the matter carefully: does your firmware need documentation? If so, what, and how much?

This blog post is the latest in a series that is longer than I expected it to be. 🙂 Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … documentation?

Why does my firmware need … a Quality Process? (2)

In my last post, I said I couldn’t write your Quality Process for you, and I can’t. That’s because your process needs to be personalised (or ‘teamalised’, even though it isn’t a proper word). Only you and your team know how you work, which is the core content of your QP document. But there is guidance, and there are hints, that I can offer.

Never ever include unnecessary detail. [This applies to all technical documents.] This guideline benefits you at least as much as it benefits anyone or anything else. There are a surprising number of justifications for this. Here are a few of them.

  • If you commit yourself to producing a design that is green (I know; just trying to keep my analogies simple), then find during coding (which is also design) that it needs to be blue, you have to go back and change your document too. Just say the design will be coloured to optimise reflectivity (or whatever), or don’t mention colour at all, unless it’s a significant attribute of your design.
  • If you specify too much at any stage or abstract level, you risk doing too much up front, (unnecessarily) constraining the design that still remains to be done.
  • Unnecessary detail takes attention away from the significant things you have to say.
  • Unnecessary detail often breaks DRY, which is something we never do. 😉

Write your QP for trusted firmware designers, not for miscreants that must be kept in order. We are all professionals; mutual respect is essential.

Never add anything to your QP that isn’t true. It will be no good to you if you do. But when you write it down for the first time, you may see that the way you work is not how you think you should work. Good! Agree the right way to work with all involved parties, change over to the new practice, and then describe it in your QP. Your QP has already benefitted your work.

Every firmware project is subject to requirements that change (or are added) during its lifetime. The development process you describe should describe how you deal with this. Agile techniques offer possible answers; I’m sure there are others.

If you always spend considerable time debugging, include that in your QP description. [Then, perhaps, you might want to give some thought about how you can spend more time on the good stuff, and less on debugging. 🙂 Dare I suggest TDD?] If you carry out, or assist in, acceptance testing, include that too. Don’t miss out any contribution you make to the development process.

When you have finished, you will have a checklist, useful to the newbie and the experienced developer alike. It will be especially helpful if you are unlucky enough to work without the support of a team. It will help your managers too.

Hope this helps?

This is an addendum to the fourth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … a Quality Process? (2)

Why does my firmware need … software patterns?

Our firmware needs software patterns as much as any software does. Which is to say we don’t need them, but they are a valuable tool. A pattern is written by a designer experienced enough to have come across many specific problems, and recognise them as instances of one general problem. Then it offers a general solution that has worked well, explaining why it might suit your specific problem, and (more importantly?) why it might not. A pattern is a valuable capsule of design information.

As a firmware designer, you will find that most (not all!) published patterns describe problems that you do not, and will not, encounter. [This applies to all users of patterns.] Inasmuch as anyone is to blame, this is our fault. It takes seasoned firmware designers to recognise and describe the problems that occur commonly in our work. If we want more firmware patterns, we need to write them…. [There are a couple of books — here and here — that look promising; I haven’t read them. If you have, please let us know what you thought of them. Thanks.]

The solutions recommended in patterns are dependent on various things. The implementation language is one of them. A pattern that is useful for one language might be unnecessary in another, whose native features automatically solve the problem, or prevent it from occurring in the first place. The solutions in the GoF patterns book are mostly aimed at a C++ implementation. Even if they apply to your firmware — some of them do! — you will probably need to translate them into your own implementation language (probably C). This is part of using patterns anyway: the solutions recommended are general ones. You need to adapt them to your particular needs.

In firmware, our implementations are constrained to be simple and simplified, because we don’t have the resources (RAM, processor power, etc.) to support the techniques and languages that many of our software peers enjoy. This simplification may lead some to think that our firmware solutions have over-simplified the pattern. Are we really using software patterns? Yes, I think we are, if our applications use solutions that are informed by the original patterns.

Here is an example from my own experience. If you think I didn’t really use patterns properly, I’m not going to argue with you. The important thing is that I found the patterns helpful in coming to a successful and workable solution to my problem. Investigate patterns for yourself, draw from them anything you find valuable, and use it.

On an OO project using C++, I designed a message handler. Our target hardware connected to its host via a serial link. When a message from the master arrived, it had to be processed. I combined the Command and Factory patterns to come up with a solution.

When a message arrived, and had been validated, its payload was passed to a Factory method that created a command instance whose type matched the message (command) just received. The Factory returned the new instance using a pointer to the abstract command type. The application then called the Execute() method of the returned instance (which processed the message as required) and then deleted it. An elegant, if simple, use of patterns to solve the problem.

Later in my career, I revisited this solution in C. We used a very simple approximation to OO that we called ‘classes in C’. Many of the nuts and bolts that an OO language would have hidden are explicitly coded in C. And so my solution comprised a message-handler class with an API method — i.e. a public class method — to which the message payload was passed. This method extracted the command type from the payload, and used a simple jump table to execute the appropriate method, and process the message. [It is worth noting here that the message-processing code is exactly the same in the C and C++ examples; it’s how you get to it that’s a little different.]

In both cases, a simple and extensible technique is used to connect a particular message with the code that will process that message. In both cases, the Factory and Command patterns inform the solution. I would say that I used patterns in my firmware. But, as I have said, I won’t argue with you if you disagree. The important thing is the solution, which stands for itself.

In a comment to a previous blog post, Matt Chernosky asked for more details on using patterns in firmware. This is your answer, Matt. Is it what you hoped for?

This is the fifth in a short series of short blog posts. Their main (intended) purpose is to stimulate discussion, so please leave a comment. Thanks for dropping by! [Find me on Twitter as @Patternchaser.]

Why does my firmware need … software patterns?