Nicholas Sheppard
Problem solving

Many treatises on skills required by modern workers—software developers and other kinds of engineer in particular—laud "problem solving", yet few schools or universities teach a course called Problem Solving. I've never been sure that such a thing as "problem solving" exists in the abstract—a few universities teach courses called Problem Solving with Python, for example, but it's not clear how Python would apply to problems in management, public policy, personal relationships, and so on. So what is problem solving and how does one become proficient in it?

Michael Martinez (1998) gives a succinct definition: problem solving is the process of moving toward a goal when the path to that goal is uncertain. In this sense, software developers solve problems every time they write a function to perform a non-trivial task, select an architecture to satisfy the needs of a system, or correct a bug.

Characterising problems

Psychologists who study problem-solving (Bassok & Novick 2012; Dunbar, 1998) distinguish two kinds of problems: well-defined and ill-defined.

Well-defined problems are those in which the starting point, the goal, and the possible steps are fully known. Computer science has developed a vast library of well-defined problems, for which good solutions are often known: put a list of objects into order, find the shortest path from point A to point B, and so on. Writing computer programs usually involves finding solutions to a large number of inter-related well-defined problems.

For ill-defined problems, however, at least one of the starting point, the goal or possible steps isn't known: I want to keep in touch with my friends, I want to write a professional-looking report without any skills in document design, I want to keep my employees happy and productive, and so on.

One might say that requirements engineering is the art of transforming ill-defined problems into well-defined ones, such that they can be solved by a computer program. Requirements should thus identify well-defined starting points, well-defined goals, and well-defined paths between the two, such that software can produce acceptable solutions to the ill-defined problem the users began with.

Problem-solving strategies

Psychologists have two distinct frameworks for analysing the way people solve problems: problem representation, originating in the 1940s, and searching a problem space, originating in the 1970s.

Problem representation emphasises finding a description of the problem from which the solution is obvious. The classic example used in psychological research is the so-called Radiation Problem, which asks how to destroy a tumour using some electromagnetic radiation without harming the healthy tissue surrounding the tumour. Someone trying to destroy the tumour might think of the problem as how to get the radiation to the tumour without passing through the healthy tissue, or how to separate the tumour from the healthy tissue, and so on, until they hit upon the canonical solution, which is to deliver low-intensity radiation from several directions such that the tumour collects high-intensity radiation while any given part of the healthy tissue receives only low-intensity radiation.

Searching a problem space emphasises starting in an initial state and finding a path to a goal state. People typically can't hold the entire problem space in their working memory—software developers would be familiar with phenomena like combinatorial explosions and NP-hardness—so they deploy a variety of heuristics for finding a path through the space. "Heuristics" differ from "algorithms" in that the first are strategies that might lead to a solution while the latter are guaranteed to find one (so deploying a well-known sorting algorithm to sort some numbers, for example, isn't really "problem solving" in this sense).

Martinez says that heuristics are often learned incidentally, rather than explicitly taught at school, which has also been my experience. He thinks that heuristics should be explicitly taught, and suggests several common ones that I think most good software developers would have picked up somewhere along their journey.

One class of heuristics is formally known as means-ends analysis, in which the problem solver tries to identify an intermediate state somewhere between the initial state and the goal state, such that finding the paths to and from the intermediate state is easier than finding the path from initial to goal in one great leap. One such strategy is "working backwards", for which Martinez gives choosing a career as an example: How do I become a software developer? I learn software development at a university. How do I learn software development? I study the subjects that the university tells me should be known by software developers, and so on.

Another strategy (not mentioned by Martinez but mentioned in other reviews) is hill-climbing, where the problem solver progressively moves from his or her current state to the next state in the direction of the goal state (the top of the hill). Both software developers and mountain climbers would know that this doesn't always work, because the climber can get stuck at a "local maximum" from which one needs to go through a valley to get to an even higher peak. But it does work for what are known as "greedy" algorithms, like the minimum spanning tree problem.

A somewhat different strategy is successive refinement, which starts by sketching a solution that resembles the goal but may have some unsatisfactory elements, then making progressive refinements until a fully satisfactory solution is found. Writers do this by writing drafts and programmers do it by writing pseudocode.

Martinez' last strategy hearkens back to problem representation: draw a diagram (formally known as an external representation). Whenever I discuss software architecture or teach software design, I make sure to have a whiteboard on hand, because the audience can digest a few boxes depicting classes or computing nodes (never mind the UML) much more easily than they can digest lines of code corresponding to the same relationships. In a software development context, I've heard this described as thinking at multiple levels of abstraction (not in the sense of abstract classes, but in the sense of high-level architecture and low-level code).

Developing problem-solving skills

Quite a few research articles discuss ways of imparting problem-solving skills to computing students, most often in the context of programming courses like the aforementioned Problem Solving with Python. In the remainder of this article I'll discuss a discipline-agnostic review of methods for teaching problem-solving recently published by Regina Frey and colleagues (2022).

While Frey and colleagues discuss problem-solving independently of any discipline, they recognise the importance of prior knowledge in tackling problems within specific disciplines. In solving programming problems, for example, computer programmers make use of their knowledge of constructs like loops, variables and functions, together with knowledge of the software libraries referenced by the program. From my own experience, programming in a new language or with a new framework always begins a little slowly, until I've acquired enough knowledge of the language and framework to fluently apply my general knowledge of software development.

Frey and colleagues describe five well-studied approaches to teaching problem-solving, two in which explicit instruction occurs before problem-solving proper and three in which problem-solving occurs before explicit instruction.

In instruction-first approaches, the teacher provides strategies for solving problems, then asks the students to apply them. Worked examples, for example, are widely used in introductory mathematics, engineering and science subjects. A good worked example demonstrates the strategy for solving a particular class of problem, such that students can readily digest each step of the process rather than having to comprehend the whole solution at once. Once shown the strategy, students can practice on new problems individually, or in facilitated groups.

In problem-solving-first approaches, students explore problems without up-front instruction, but are guided through the problem space by a facilitator. The contrasting cases approach presents several problems that differ in key details, illustrating important features of the problem space. The productive failure approach presents students with problems that they aren't expected to know how to solve, then guides them on how overcome the problem (making it a kind of backwards worked example, not simply "sink or swim").

Conclusion

As defined by psychologists, problem solving does exist, and it can be taught. Possessing an arsenal of heuristics is a good start, but significant knowledge of the problem domain is often necessary to chart a path through a given problem space.

References


Miriam Bassok and Laura R. Novick (2012). Problem Solving, in: The Oxford Handbook of Thinking and Reasoning, edited by Keith J. Holyoak and Robert G. Morrison, Oxford University Press, Oxford, 2012, chapter 21.

Kevin Dunbar (1998). Problem Solving, in: A Companion to Cognitive Science, edited by by W. Bechtel and G. Graham, Blackwell, London, 1998, pages 289-298.

Regina F. Frey, Cynthia J. Brame, Angela Fink and Paula P. Lemons. Teaching Discipline-Based Problem Solving, CBE—Life Sciences Education, volume 21, number 2, 2022.

Michael Martinez (1998). What is Problem Solving?, The Phi Delta Kappan, volume 79, number 8, April 1998, pages 605-660.