Developing Learning Objectives: Systems Theory and Tools

This post is a bit longer than I plan for a Blog post and is in response to the Poll question and some comments regarding difficulty with developing objectives. It may take some uninterrupted time to make the best of it.  But I hope you find it valuable.

Learning Objectives

The purpose and function of a learning objective was covered in the post, Learning Objectives - The Rosetta Stone of ISD. This post will detail the process of developing sound learning objectives that will serve you well as you design course content.

Defined

Learning objectives are statements that describe what the learner is expected to achieve as a result of instruction including the specific behavior, the conditions under which the behavior is to be performed, and the criteria for successful achievement of the objective.

Each part of a learning objective is like a leg on a three-legged stool.  In order for a learning objective to function correctly, you need all three parts.  Otherwise… thud!

An Objective By Any Other Name

Objectives go by many names; behavior objectives, instructional objectives, learning outcomes. Regardless of what you call them they serve the same purpose as defined above.

Note: This is different from a goal which has a broader, less specific intention.

  

Developing Objectives -Determining the Behavior

The method used to identifying behaviors depends on the primary source of information for your instructional design. I propose there are two types of sources: Subject Domain and Performance Based. How the behavior is determined is very different.

How to Determine the Behavior in a Subject Domain

The subject domain is the one that requires the exercise of more judgment on the developer’s part.  In our information rich society, the amount of information in a topical area can be overwhelming; making it easy to put the student into information overload.

Look back at your education; a person or body of authority decided what would be taught.  And, I am sure you have experienced at least one class where your interest level was low and probably thought the content had little or no relevance.  I have little to offer on how to identify the content; although I can offer a process for identifying the desired behavior.

The steps to determine the Behavior in a Subject Domain include:

·       Define the content

·       Determine the type of learning outcome

·       Choose a level of achievement

·       Choose the behavior (verb)

Each step will be covered in more detail.

Define the Content

The information to be included may be broad or very specific.  For example, think about the domain of history.  The content could be broad like a survey of the history of civilization or as specific as the history of the Alphabet Houses of Richland, WA.  Now it’s your job to decide.

This decision may be influenced by the amount of time allotted for teaching and how much information can be reasonable covered in that time.  Your task is to define the content.

Determine the Types of Learning Outcomes

Once the content is identified, the appropriate type of the learning outcome(s) must be chosen.  Learning can be divided into three areas of outcomes: affective, cognitive and psychomotor.  

For the purposes of this post, I am going to assume you are familiar with the types of learning outcomes. At least from the Basic Trainer’s Training Course.

If not, Cindy Vinson gives a good overview and, there are details on Bloom’s Revised Taxonomy of the Cognitive Domain by Iowa State University.

Choose a Level of Achievement

Each type of learning outcome has increasing levels of complexity (note: not difficulty). For example, Bloom’s Taxonomy of the Affective Domain begins with receiving and ends with internalizing.  It is like a catalog that offers several versions of the same product with different quality; you might remember the old Sears good, better, best approach.

Levels of achievement for the Affective Domain of Learning Outcomes

Your decision will be influenced by abilities of your potential audience and how it aligns with other learning goals.  Usually, the levels of complexity build on each other.  For example, in order to achieve behaviors at the “Respond” level you must have acquired the “Receive” behaviors.  Your task is to choose which level of complexity to teach to.

Choose the Behavior (verb)

Each level of complexity has a set of verbs that describe the behaviors. Think of it as a thesaurus for that level. In the affective learning outcomes, the “Receiving” level refers to the behaviors of: awareness, willingness to hear, and selected attention. 

Your task is to decide which verb most closely matches the behavior you desire your students to achieve.

 Verbs Related to the Receiving Level in the Domain of Affective Learning Outcomes

Pitfalls

As you have probably experienced, what a designer chooses may not be of interest or relate to you; definitely a concern in the world of adult learning. There is a tendency for Subject Matter Experts to want everyone to know what they know and the best service we can do as instructional designers is to temper that tendency.

How to Determine the Behavior from Performance Based Source

The other method for determining the desired behavior is to use the results of a job analysis. I define a job analysis as the process used to collect information about the duties, responsibilities, necessary skills, outcomes and work environment of a particular job. The result of a job analysis is a list of the tasks that are performed.

Task statements are short sentences containing the actions performed and the objects involved in the performance.  Yep, action = behavior.

The list is then refined to determine which tasks will be trained on called a “task-to-training” matrix.

How to write a task statement is outside the scope of this post. The Department of Energy provides some guidelines in theri Training Program Handbook, A Systematic Approach to Training (DOE-HDBK-1078-94), Attachment-2, as well as, Kenneth Carlisle’s book, Analyzing Jobs and Tasks.

Here is an example task statement from the Training Specialist job: Boot-up computer and projector system. The action is “boot-up;” the objective behavior is “boot-up”.

I hope the mushy thought producing organ between your ears has just formed the exclamation WOW! No guessing, no assuming, the behavior that is actually performed on the job has the potential of being of value to the learner.

Determining the Condition

The condition in a learning objective is a description of the environment where the behavior is to take place.  The description should include any items or equipment necessary to perform the behavior. Condition statements contain conditions that either aid or limit. An aiding condition advises the learner of things that will be available to assist in performance of the specified behavior. Examples include, Given a list of options …; Using a calculator …; or In the Chemistry laboratory …

A limiting condition advises the learner of limitations that are likely to make the action more difficult to perform. Examples include, From memory state …; Without the use of a procedure …; In the absence of a computer …

In either case, the condition statement needs to be relevant to the performance of the action and should include only what will be provided or denied during performance evaluation.

Subject Domain Conditions

Conditions for learning objectives developed from a subject domain have a tendency to be restrained by the existing facilities. For example, the classroom. It is the exception that equipment or facilities will be purchased to support an objective so a level high in the taxonomy hierarchy can be achieved.  Think of it as the objective level is set so it can be accomplished in the current conditions.

Performance Based Conditions

Conditions are derived from information collected during task analysis. They also include equipment, tools, and references necessary to perform the task. The learning environment should represent the work environment as much as practical. In the world of work millions of dollars have been spent on simulators and equipment. HAMMER is a good example of having functional props for training.

Inferring a Condition

By convention, the condition “from memory” can be implied rather than specified. Using implied condition statements has an advantage of avoiding redundancy when the conditions are identical for a set of learning objectives.  Include relevant conditions for all other cases.

Determining the Criteria

The criteria part of a learning objective defines the degree of mastery or accuracy the behavior is to achieve.  It answers the question, how good is good enough? 

The standard will refer to the quality of the end product or the precision of the process and is usually expressed in terms of time limits, accuracy, quality, or quantity.

Subject Domain:

These criteria usually correlate to the level within the hierarchy of the taxonomy. In general, the lower level in the hierarchy, the less subjective the criteria. The Remember this level of the taxonomy in the Cognitive domain usually deals with facts and the criteria is usually 100% accurate. For example, Fact: The year Columbus sailed for the Americas (1492).

At the other end of the hierarchy is Evaluation and the designer must exercise judgment as to what is important and to what degree. Should the criteria focus on the final evaluation product or the process used during the evaluation? Is the scale used for evaluation (which should be supplied in the conditions), dichotomous or a relative scale? If it is relative, what is an acceptable degree of accuracy?

Performance Based Criteria

The best guide is to define the consequence for poor or incorrect performance of the target behavior. In general, the less acceptable the consequences the higher the criteria.

The criteria for an objective should be derived from the standard required by the task and closely approximates actual performance standards on the job.

Inferring a Criteria

By convention, the criteria, “100% accuracy,” can be implied rather than specified. Using implied condition statements has an advantage of avoiding redundancy when the conditions are identical for a set of learning objectives. Include the relevant criteria for all other cases.

Example of a learning objective

Given a task-to-training matrix develop a learning objective that includes: a behavior, conditions and criteria.

Levels of Learning Objectives

I have experienced designers that really get hung-up on the hierarchy of learning objectives.  To me it becomes a matter of practicality.  I don’t care what you call them it comes down to which one is a prerequisite in order to achieve the next one; an important consideration when designing instruction.

Terminal Objective:

Some designers believe there should be only terminal objective.  It is more akin to a goal and can be used as the global picture or final goal of a course.

The DOE Handbook ⁽²⁾ indicates “They are translated directly from the task statement, and provide the framework for the development of training/evaluation standards, enabling objectives, and lesson plans.”   This would mean there is a terminal objective for each task.

Enabling Objectives

Enabling objectives are a prerequisite behavior that must be achieved in order to master a terminal objective. The DOE Handbook ⁽²⁾ indicates enabling objectives are ”derived” from the knowledge and skills identified during task analysis”.

Subordinate Objectives

When necessary, subordinate objectives are prerequisite behaviors that must be achieved in order to master an enabling objective. The DOE Systematic Approach to Training does not address subordinate objectives; although they are used extensively in the Dick & Carey⁽³⁾ systematic design of instruction model.

Summary and Conclusion

Remember for the instructional designer, learning objectives are a tool and the key to instructional development, design and evaluation. In my experience, it is worth the effort to be as rigorous as possible when developing them. It just makes the rest of the design process focused and easier. You may wish to review the Objectives, the Rosetta Stone of ISD post.

Best,

Cj

REFERENCES:

1.   Carlisle Kenneth, E. (1986). Analyzing Jobs and Tasks. Educational Technical Publications. Englewood Cliffs, New Jersey.

2.   Department of Energy - DOE-HDBK-1200-97 Guild to Good Practices for Developing Learning Objectives.

3.   Dick, W., & Carey, L. (1996). The systematic design of instruction (4th Ed.).  New York, NY: Harper Collins.

Learning Objectives – The Rosetta Stone of ISD

Theory Basis:  Instructional Theory

The Rosetta Stone was discovered in Egypt in the late eighteenth century, inscribed with ancient Egyptian hieroglyphics and a translation of them into Greek. The stone proved to be the key to understanding Egyptian writing.   Today we use the term to mean, a clue, breakthrough, or discovery that provides crucial knowledge for the solving of a puzzle or problem.  Learning objectives are the Rosetta Stone of ISD because they solve the puzzle of how to progress through a systematic approach to training.

The original discoverer of the stone had a goal in mind; and goals vary.  So is the case in the arena of ISD.  The use of an objective depends who is using it.

For the Instructional Designer

Objectives solves the “How” to take the results of an analysis to making decisions about choosing an instructional method and provides the tool to constructing a valid evaluation instrument.

For the instructional designer, an objective should be a meticulously developed statement based on a task selected for training from an analysis that describes the desired behavior (action verb), conditions, and achievement criteria of a desired student performance (mastery).

The three parts of an objective used by a designer helps to guide the plethora of decisions involved in designing training.  See the chart below:

For the Learner

Objective solves the “What” is this session all about.  It is the key to awakening previous knowledge on the topic, provides guidance on choosing a learning strategy, and establishes a clear endpoint for successful completion.

In addition, objectives allow the learner to make a judgment regarding the value of the session; influencing the level of motivation to apply.

Use and Abuse

The general principle of sharing the learning objectives with the learner is a sound one.  Unfortunately, some historical events have yielded a poor practice; sharing a list of objectives written for the instructional designer at the beginning of a course.  Let’s face it there is nothing motivating about reading or listening to someone read off a litany of objectives that can’t be understood. This method may actually interfere with learning.

There are multiple sources that will bear out the following; although I think Smith & Ragan ⁽¹⁾ say it best, “Only rarely will designers express the objective to the learners in the same forms that were used when designing instruction”.

Objectives shared with the learner should be understandable by the uninitiated, shared prior to the related content, and describe how success will be achieved.

Quick Quality Check: Ensure terms that have not been learned yet are excluded from the objective shared with students.  Ask an inexperienced person to explain what will be learned and a description of how they would go about achieving success.

Challenge:  Choose a lesson and delete the list of learning objectives at the beginning of a learning session.  If not already existing, replace it with a goal statement describing how the content will be applied.

In my opinion, there is an overuse of the action verb “identify”.  Recently as part of a group we were brainstorming some instructional methodologies.  In reviewing the objectives; eleven of fifteen objectives used the action verb “identify”.  Humans identify something by using our senses; generally our vision.  Examples from the course we were reviewing:  Identify a:  method, route, components, limits, strategies … and the list goes on.  Unless you’re some type of bio-robot doing a quality check as objects pass by, most of us don’t stand around and identify stuff.

I submit there might be a bit of confusion with some knowledge of factual information required to accomplish a task and gets interpreted as the ability to identify it.   The task may be to apply a method, choose a route, operate a component or adhere to a limit.

The concern is that if you are focusing on the wrong tasks your training will be valueless on the job.

Challenge:

Review the objectives for one of your learning sessions, select the objectives using the action verb “identify” and do the following reality check.  Ask yourself, if this really the action you anticipate the learner to do in the work place.  Follow this up with a review of the evaluation; is the verb in the evaluation congruent with the verb in the objective and task?

The Missing Link:

The key to a valid evaluation is the action verb in the objectives; it should be the same ones used in the evaluation.   For an individual test item (item analysis) the link is called “Item-Objective Congruence.  Ronald .A. Berk ⁽³⁾ describes the relationship as “The most important (test) item characteristic.”

The quality of your evaluation instrument is directly dependent on the degree of agreement between the action verbs in your objectives and the actions in the evaluation used to measure the mastery of the objective.

Quick Quality Check:

 The verb in the task statement and the verb in the objective should be the consistent.  The verb in the objective and the verb in the evaluation instrument (read test question) should be congruent.

Your Turn

Your comments, candid and kind, will be appreciated and the next blog posting might depend on the answer to the poll question.  Please take time to answer it. (Analogy)

Regards,

Cj

Resources:

1)   Smith, P. L. & Ragan, T. J. (1993) Instructional Design. Upper Saddle River, NJ. Prentice-Hall.

2)   DOE-HDBK-1200-97, Guide to Good Practices for Developing Learning Objectives.

3)   Beck R. A. (1982). Criterion-Referenced Measurement: The State of the Art. Baltimore, Maryland: John Hopkins.