Tim Cliffe - Blog

Identifying Learning Content - What You Don't Know You Don't Know

01 Target Audience

(01.1) Anyone interested in Training Needs Analysis or content identification for Instructional Design.

02 Executive Summary

(02.1) When identifying content for learning resources, it is important to identify all relevant knowledge for inclusion. However, some knowledge (information) is difficult to identify.

 

(02.2) This article presents the application of the well-known model, The Four Levels of Teaching (Martin M. Broadwell, 1969)¹, to the task of identifying potentially important instructional content (Training Needs Analysis (TNA)) when those involved in the learning resource development do not know such content exists.

03 Structure of This Article

• (04) The Four Levels of Teaching
• (05) Adapting The Four Levels of Teaching for TNA
• (06) What You Don't Know You Don't Know
• (07) Identifying What You Don't Know You Don't Know - Scenario
• (08) Summary
• (09) References

04 The Four Levels of Teaching

(04.1) In 1969, Broadwell¹ was searching for a new model to describe knowledge states. He identified The Four Levels of Teaching:

1. Unconscious Incompetence;
2. Conscious Incompetence;
3. Conscious Competence;
4. Unconscious Competence.

The Four Levels of Teaching are the states of knowledge describing a person's progression from 'ignorance' of a subject to 'mastery'. Note: The model is often, incorrectly, credited to Maslow.

 

(04.2) The image of The Four Levels of Teaching deliberately avoids showing the progression in a fixed order. Although the process of knowledge states often progresses from 1 to 4, as listed above, a particular individual, at a particular point of learning, may 'jump' between states as various aspects of knowledge come to their attention.

05 Adapting The Four Levels of Teaching for TNA

(05.1) Broadwell's¹ model, however, describes more than knowledge states. It is easily adapted to support the efficient and exhaustive identification of the learning content required to enable progression through those states.

 

(05.2) This requires a simple re-writing of the original, to move the focus from 'teaching (learning) states' to 'knowledge requirements'.

 

(05.3) The figure shows the adaptation of the model to change its focus to 'knowledge requirements'. The four stages directly correlate with Broadwell's¹ original model.

 

(05.4) When conducting a TNA, the concern is to identify all required knowledge to enable the student to progress to 'mastery'. In Broadwell's¹ model, this is described as 'Unconscious Competence'. In the adapted model, this is described as 'You don't know what you know'. Now we are left with just three requirements to complete the TNA.

 

(05.5) From the perspective of a TNA:

• 'You know what you know' is a great situation. This knowledge is known and easily identifiable;
• 'You know what you don't know'. This knowledge isn't known, but you know you don't know, so you can find out.

But what about 'You don't know what you don't know'? This is a little more problematic. How do you find out, what you don't know you don't know?

06 What You Don't Know You Don't Know

(06.1) So, how do you find out, what you don't know you don't know?

 

(06.2) To illustrate, I will use an example based on an algorithm developed by the National Clinical Governance Group of the UK National Health Service (NHS). The algorithm is called NHS Pathways
(External link, opens in a new tab/window). Pathways.

 

(06.3) The reason for choosing this example is simple. Few readers will have experience of Pathways, consequently, the underlying principle of the example, given below, is less likely to be clouded by 'familiar territory'. This, in turn, will encourage the reader to apply the principles discussed, not the example.

 

(06.4) NHS Pathways is a telephone triage tool, designed by clinicians, that enables staff, without clinical training, to triage over the telephone. The underlying principle of Pathways is not to diagnose (telephone triage call-handlers are not clinically qualified), but to rule-out potential diagnoses, and sign-post a caller to the most appropriate clinical intervention, available at the time.

 

(06.5) NHS Pathways includes supporting information, which, together with the skills of an experienced call-handler, enables a call-handler to identify any important information the caller may not have declared. In other words, to identify 'what the call-handler doesn't know they don't know'.

07 Identifying What You Don't Know You Don't Know - Scenario

(07.1) Pathways is so-called because it uses a large number of clinical pathways appropriate to, for example, gender, age (infant), illness, injury blunt, injury penetrating, and so on.

 

Scenario

(07.2) Imagine a parent has telephoned about their three-year-old daughter who suffered an unwitnessed fall from her cot, after being put to bed.

 

(07.3) Your initial Pathways triage has established the child has not sustained life-threatening injury. You confirm the female infant has suffered a blunt injury, sustained during an unwitnessed fall from her cot.

 

(07.4) Remember, just as with a TNA, the home is as familiar to a parent as the working environment is familiar to a master practitioner.

 

(07.5) What has the parent taken for granted? What haven't you been told? What makes a difference in this scenario? What is it you don't know you don't know?

 

(07.6) Identifying what you don't know you don't know is achieved through a combination of:

1. Visualisation;
2. Applying Your Existing Knowledge;
3. Probing Questions;
4. Intuition.

Visualisation

(07.7) In your real-life situation, you will be unfamiliar with the subject at hand. There are things you don't know.

 

(07.8) I will assume it is known what the subject, of the training resource, looks like. Either because it has been visited (in which case, the methodology, described below, can be used in-situ) or suitable video footage or photographs have been seen. This will be necessary to enable effective visualisation.

 

(07.9) In this scenario, you're on the telephone, you can't see. In your mind, walk into the little girl's bedroom:

1. What might you see?
2. What may be important that cannot be seen?
3. What does seeing the child climb out of the cot and falling look like?
4. What could be different to what you imagined, when you first walked through the door?
5. Is something there now, that wasn't before. Perhaps something is missing that was present during the fall?

Applying Your Existing Knowledge

(07.10) In your real-life situation, you will need to use what you do know to help you identify what may not be known.

 

(07.11) In this scenario, you need to use existing knowledge to guide you:

1. Where is the cot (close to a wall, door, window, or furniture)? Could the child have hit something as she fell?
2. What is on the floor (carpet, rugs, laminate, concrete, toys, small tables)? Could something on the floor have also injured the child? Landing on a rug on a carpeted floor is very different to landing on a laminate flooring on a concrete floor;
3. Are there any indications as to what happened as the child fell (marks on walls or furniture), or what the child landed on (broken toys, scattered items)?
4. Does the child have any marks that may indicate which parts of the body were impacted (cuts, bruising, a bump on the head)?
5. Has anything been moved or added, since the fall, that may be important? Has the side of the cot been lowered, where was it when the child fell? This will give a good indication as to the height of the fall. An infant falling from above 1 metre is more serious than falling from less than 1 metre;

Probing Questions

(07.12) The use of probing questions will help you complete the image in your mind, and discover what you do not know.

 

(07.13) Pay careful attention to the answers given. There may be additional information, or opportunities for further probing.

Intuition

(07.14) There is no methodology or science to describe intuition. This does not mean it has no place in discovering the unknown.

 

(07.15) We have all experienced those moments when we know, somehow, we are missing something, or something doesn't quite fit. We all know those "I can't put my finger on it, but..." moments. They are often very reliable, listen to yourself, don't dismiss them.

 

(07.16) The worst case is, you are wrong and no harm is done. Alternatively, you may identify something that makes a real difference.

08 Summary

(08.1) As stated earlier, the scenario above has been deliberately chosen to convey the principles involved in discovering 'what you don't know you don't know' and how this can contribute to identifying important knowledge, or understanding, of the situation at hand.

 

(08.2) In my experience, I have discovered what were, in the beginning, unexpected benefits.

 

(08.3) For example, in one circumstance, the above method identified knowledge that was possessed by very few people (all in the same 'isolated' setting), and had never been documented, or shared. A reference to the knowledge was included in introductory training materials, and then added to the design requirements for a later, more advanced, coverage of the topic. Consequently, the knowledge was distributed and also protected from being lost, if for example, the few who originally possessed the knowledge left the organisation.

09 References

1. Broadwell, Martin M. (1969)
(External link, opens in a new tab/window). Teaching for learning (XVI). Accessed 2019-09-26.

Use of this Article

Any part, or all, of this article may be linked-to or copied for non-commercial purposes. Any linked or copied content to include the following...

 

Identifying Learning Content - What You Don't Know You Don't Know by Tim Cliffe © 1997-2024.

 

Where use will be for commercial purposes, seek authorisation, including details of proposed use, via the Contact page.

 

Return to the Blog page.