Theme 1 Introduction to psychological and performance assessment (1).ppt

psychology

2. APPLIED PSYCHOLOGICAL AND PERFORMANCE
ASSESSMENT AND PROFESSIONAL ETHICS
THEME 1: INTRODUCTION
Prof Aletta Odendaal

3. 3
I/O PSYCHOLOGY
I/O Psychology: the application of psychological principles, theory and
research to the work setting (Landy & Conte, 2004)
Interested in understanding, predicting and influencing behavior related to
work
Focus on individual differences related to work performance
WHY ASSESSMENTS?
DEBATE THE SCIENTIFIC MERIT OF PSYCHOLOGICAL ASSESSMENT

4. GUIDING QUESTIONS
What is a psychometric test?
What is a psychological test?
Why do we use psychological assessment?
Reasons for testing and not testing?
Describe the categories of psychometric tests
Describe the statutory control of test classification, test use and test
standards in South Africa
What is fair and ethical assessment practices?
What are the roles and responsibilities of test users and test developers
respectively?
What are the roles and responsibilities of test users and test takers with
regard to computer and technology based testing?
What are test takers rights and responsibilities?
4

5. NATURE OF TESTS
•Huysamen (1989) – Standardised procedure for obtaining and assigning
numbers to individual’s responses to a sample of tasks
•Anastasi & Urbina (1997) – An objective and standardised measure of a
sample of behaviour
•Foxcroft et al. (2004) – Purpose of test results in psychological act (p.2-4)
Preference for “Assessment Measure” – characterised by prescribed
administration procedure; focused on a domain of functioning; carefully
controlled; systematically scored; specific interpretation procedure;
supported by evidence; designed for specific context
•Foxcroft & Roodt (2013) – Psychological Assessment aimed at gathering
wide array of info by using psychological assessment measures and info
from other sources, subject it to evaluation and make a decision on the
basis thereof – testing one of the key elements

6. USES OF ASSESSMENT MEASURES
•Assess abstract psychological constructs/aspects of human behaviour
•Diagnostic purposes (scholastic tests/psychopathology)
•Developmental purposes (counselling/career development)
•To direct interventions (therapeutic)
•Selection, training and development
•Specialised assessments (neurological injury)
•Research

7. ASSESSMENT MEASURE VARIATIONS
•Administration procedure (group vs individual)
•Whether speed plays a role/time limits (speed vs power)
•How they are scored (objectively vs subjectively)
•How the are normed (comparison group vs criterion)
•For what purpose it was developed (diagnostic etc)
•What is the nature of the items (verbal, performance)
•What kind of response is required from the test-taker (pencil-and-paper etc)
•Which content areas are tapped (ability, personality etc)

8. Benefits when assessments are applied
effectively:
Cost of selection errors:
•Recruitment & selection
•Training & development
•Poor performance
•Individual stress
•Impact on team & individual
morale
The objectivity of testing:
•Standardisation & fairness
•Comprehensive & detailed data
•Subjectivity of interviews
The validity of testing:
•Relation to job inherent
requirements
•Link to predicting job
performance / success
•Cost-benefit
Public relations:
•Efficiency of testing groups
•Professional and committed
employer
•Objectivity of multi-method
approach

9. What can go wrong?
Bias against
a specific
group
M
isinterpretation
of results
Negative stigma
regarding testing
Unfair
decisions
Inappropriate
measures are used
Risk to organizational
reputation
Cost of
litigation
Leakage of items to
the public
Psychological harm to
the individual
Inconsistent
results

10. “Metrics only have value to the
extent that they change vital
decisions about talent”
1
Wayne Cascio, SIOP Chicago, 2011

11. Reasons for NOT testing
Costs
Fear of legal action
No qualified people employed to control tests
Practical constraints (location and numbers)
Current decision-making process would not
improve upon by using test

12. What is a psychometric test?
Developed according to psychometric test theory.
Specific procedures followed during the
development.
Adheres to specified quality control standards:
- administered in a standardised manner
- scored in a standardised way
- interpreted according to a standardised format
- constructed according to psychometric
principles

13. Constructed according to……
 A theoretical rationale
 Writing of experimental items
 Piloting of experimental items
 Item analysis
 Standardisation
 Validation
 Technical write up

14. What is a Psychological test?
A structured assessment measuring a psychological
construct
Who decides if it is a psychological test?
The psychometrics committee of the Professional Board for
Psychology (HPCSA)

15. Categories of psychometric testsCategories of psychometric tests
Maximum performance
–General ability: Fluid vs. crystallised
–Specific ability: Mechanical, spatial, visual acuity,
perceptual speed and accuracy etc.
Typical performance
–Personality
–Interests
–Values

16. Learning Potential
Individual’s ability for achievement in a particular area
1.Crystallised ability: the extend to which an individual have
benefited from past experience and education.
Verbal and Numerical Reasoning
2.Fluid ability: extend to which an individual will benefit from
future experience and education
Abstract reasoning (Non-verbal )
3.Learning potential: typically involves a test-train-retest
strategy with some form of training provided as part of the
assessment process
Dynamic Assessment

17. Assessment applications in broader talent
management practices
Development
Selection
Restructuring
Outplacement
Research
Training
Career
Counselling
Succession
Planning
Assessments

18. Applications of testing
1.Typically for selection and development
2.Remember reliability of information from different sources depends
upon type of information needed:
INTERVIEWS
Low: Personality, ability, aptitude
High: Self-presentation, social skills, technical knowledge
CV
High: Experience and qualifications
Dubious: Technical knowledge judged from experience
TESTS
High: Ability, aptitudes
Moderate to high: Personality, integrity, values and motives

19. General Guidelines:
High stakes: where a third party requires the
results of the test for use in process of making an
important decision about a test-taker
Low stakes assessment: where the test-taker
obtains the information for his/her own
development/interest.
1
9

20. Some facts about testing
Well constructed psychometric tests predict
performance effectively and economically
Relevant objective tests are better predictors
of performance than almost any other single
assessment method

21. Assessment Process: ISO Aligned

22. Two approaches to the regulation of supply:
1.Education focuses on providing potential test users with the knowledge
and skills they need to be able to use tests appropriately and safely:
This knowledge and skills is partly a function of the user (e.g.
through test user qualification training) and partly related to the
instrument.
Test reviews provide users with valuable, independent judgements
about the strengths and weaknesses of tests
2.Restriction focuses on attempts to control access to tests such that
only those who are competent to use them can obtain them.
NOTE: The education and restriction approaches are not mutually
exclusive
(Bartram, 2006, 2008, 2012)

23. Assessment context: Business, Legislation, and
Regulation
Adapted from PAI presentation, Dowdeswell, 2013
External professional Conditions
Internal/intra professional
Conditions

24. Test Use in South Africa
Health Professions Act 56 of 1974 (Amended Act 29/2007) –
Section 33 (1) provides that Minister may by regulation define the
scope of practice of registered people in terms of the Act.
Section 37 repealed and replaced by Regulation 2 (a)-(i) (R993 in
Gazette 31433) published 16 September 2008 - restricts use of
psychological assessment measures to registered psychology
professionals.
Labour Law: Constitution of the Republic of South Africa (Act 108
of 1996), The Labour Relations Act (66 of 1995), and the
Employment Equity Act (55 of 1998).

25. Statutory control: Test User
Health Professions Act 56 of 1974 Section 37
Repealed and replaced by Regulation 2 (a) to (i)
Use of a psychometric measuring device, test,
questionnaire, technique or instrument that assesses
intellectual or cognitive ability or functioning, aptitude,
interest, personality is constituted as being a
psychological act.
Conclude: Only registered psychologists are
permitted to perform psychological acts which, in
relation to evaluation, testing and assessment are
defined in Regulation 2 (R993/31433).

26. Role of Professional Board
Main function: Protection of the public
Members of public contact Board if they feel assessment
practitioners misused assessment measures, treated
them unfairly or unprofessionally
Main function: Guide the profession
Lay down training and professional practice guidelines
and standards – ethical code of professional conduct

27. Additional Information
2
7
HPCSA:
Form 207 (List of classified psychological tests)
Form 208 (Policy on classification of psychometric
measuring devices, instruments, methods, techniques).

28. Statutory control: Test Standards
To ensure that discrimination is addressed within testing domain
Section 8 of EEA (55/1998) as amended in 2013 refers to:
Psychological testing and other forms of assessment of an employee
are prohibited unless the test/assessment being used:
• Has been scientifically shown to be valid and reliable;
• Can be applied fairly to all employees;
• Is not biased against any employee or group;
• Has been certified by the Health Professions Council of South Africa,
or any other body which may be authorised by law to certify those
tests or assessments.
Amendment retracted

29. Statutory control: Test Classification
Classification: process whereby decision is made regarding nature of a measure and
who may use it.
Psychological or non-psychological measure
Determine category of practitioner who may administer, score, interpret and
report on measure
Who is responsible for classification: Psychometrics Committee of the Professional
Board. Prior to 1996: Test Commission of SA – Level A (achievement tests) B (group
test of ability and interest) and C (individual intelligence and personality tests)
Simplified classification system: Measure is classified as either a psychological
measure or not:
When it use the results in the performance of a psychological act
Test content and outcome may have negative psychological consequences for
individual

30. Psychometrics Committee
•Mandated to control and regulate psychological test use in SA
•Actively participated in the development of internationally
acceptable standards for test use in conjunction with ITC
•Developed competency-based training guidelines for all registration
categories
•Implement (2000) and manage a national examination for different
categories of psychological assessment practitioners under the
auspices of Professional Board (Main Function: Protection of the
Public)
•Lay down professional and ethical standards in testing
•Publish an annual list of measures that have been classified and in
the process of classification

31. List of Classified & Certified Psychological Tests
64 tests that have been classified and reviewed.
133 tests that have been classified but not reviewed.

87 tests classified as “under development”

32. Why Test User Standards are needed?
•Little Uniformity in test user qualifications and standards
•Globalisation and increase in mobility of test users
important to define benchmark standards for test quality
an competence in test use
•Advances in technology and development of
sophisticated testing solutions bring alternative forms of
administration (computers and Internet)
•Require international standards of test use including
copyright, test security, ethical test use and good
practice in test administration

33. The Employment Equity Act & Professional Board for
Psychology
Section 8 of EEA Act has a broader definition of assessment than the current test classification policy of the HPCSA
Implications:
Overlooks ‘other similar assessments’ are not psychological in nature, and that Health Professions are not the only that use assessment tools
and procedures
All assessment measures used for selection, training and development purposes in employment settings must be certified that they meet
minimum psychometric requirements
Implications:
Overlooks ‘other similar assessments’ are not psychological in nature, and that Health Professions are not the only that use assessment tools
and procedures
All assessment measures used for selection, training and development purposes in employment settings must be certified that they meet
minimum psychometric requirements
33

34. Best Practice Guidelines
What are the roles and responsibilities of test
users and test developers respectively?
What are the roles and responsibilities of test
users and test takers with regard to computer
and technology based testing?
What are test takers rights and responsibilities?.
3
4

35. International Guidelines in Test Use
•International Guidelines for Test Use (ITC)
•European Test User Standards for test use in work and organisational settings
(EFPA and EAWOP)
•The Australian Psychological Society (APS) Supplement to guidelines on the use
of Psychological Tests
•The British Psychological Society (BPS) Level A and Level B standards for
occupational test use (Bartram, 1995, 1996)
•Psychological Testing: A User’s guide (BPS)
•The American Educational Research Association (AERA), American
Psychological Association (APA), & National Council on Measurement in
Education (NCME) (1985) Standards for Educational and Psychological Testing.
•American Association for Counseling and Development (AACD) Responsibilities
of Users of Standardized Tests (Schafer, W.D, 1992).
•The CPA (Canadian Psychological Association, 1987) Guidelines for Educational
and Psychological Testing.

36. South African Guidelines in Test Use
•Guidelines for the validation and use of assessment
procedures for the workplace (SIOPSA/PAI) - 2005
•Code of practice for psychological and other similar
assessment in the workplace (SIOPSA/PAI) - 2006

37. CORE COMPETENCIES: PSYCHOMETRIST
To ensure that a competent test user will use tests appropriately,
professionally, and in an ethical manner, paying due regard to the
needs and rights of those involved in the testing process, the
reasons for testing, and the broader context in which the testing
takes place.
This outcome will be achieved by ensuring that the test user has the
necessary:
competencies to carry out the testing process, and
the knowledge and understanding of tests and test use that inform
and underpin this process.
(ITC Guidelines for Test Use)

38. European Context
Use of psychological tests is restricted to psychologists (Italy)
Use of tests by non-psychologist professionals is recognised based
on standards and qualifications in test use (UK, Norway and
Sweden).
Broader qualification in occupational assessment: Germany (based
on the DIN33430 standard)
Differentiate between instruments psychologist and non-
psychologist may use (Finland)
Germany and Finland qualifications focus on knowledge. No
requirement for the assessment of competence or application of
knowledge in practice.
UK, Sweden and Norway adopt a competence- based approach:
require potential test user to demonstrate knowledge, skill and
understanding.

39. European Test User Standards
3
9

40. Assessment Strategy

41. 4
1
Questions?Questions?
Aletta OdendaalAletta Odendaal
[phone](021) 880-3001
[email protected]
Petro van ZylPetro van Zyl
[phone](021) 880-3012
[email protected]

42. PRESCRIBED TEXTBOOK
4
2

43. COURSE OBJECTIVES
After completing the course in Applied Psychological and Performance
Assessment and Professional Ethics the student will be able to demonstrate the
following competencies:
A basic understanding of the psychometrics that define the parameters of the
professional use of assessment techniques;
The appropriate contextual knowledge of the impact of irrelevant sources of
test variance and the skills to promote fair assessment practices;
With respect to selected assessment techniques, demonstrate the required
level of instrumental knowledge and skills that apply to all aspects of the
assessment process;
The appropriate communication skills to influence assessment policy and to
report assessment results within a professionally and ethically attuned work-
setting.
4
3

44. COURSE CONTENT
Introduction to Applied Psychological and Performance Assessment
Characteristics of a Good Test
Unsystematic and Irrelevant Variance in Test Scores
Guidelines for the Selection of Appropriate Assessment Techniques
Cultural Bias in RSA Testing
Administration of Psychological Assessment Measures and the
Communication of Results
Measures of Cognitive Abilities
Measures of Personality
Measures of Interests, Values and Attitudes
Computerised Testing
Measures of Actual Performance
Testing of Potential
Professional Ethics
4
4