test for Meter Reader analysis and evaluation

Meter Reading

1. ABILITY TESTING FOR
METER READERS
City of Lakeland, FL

2. Historical Information
20 Meter Reading Positions
67% turnover from FY 2002 –2005
95% for FY 2005
Most common reasons for dismissal
-Meter reading inaccuracies
Most common reasons for resigning
-Too physically demanding
-Better job, better pay
Test had not been updated in six years
New manager that understands and supports
testing

3. KSAO Linkage
Listed all KSAOs and identified what
selection components were needed
Linked each KSAO to the proper selection
component(s) (handout)

4. SelectionComponents
New Written Test
Field Test (to be developed)
Interview
Previous Employment Check
Medical Examination
Driver’s License Check
Probationary Period

5. Job Analysis Data -Tasks
Reads meters with little or no error
Enters meter readings with little or no error
Drives city vehicle up to 100 miles per day
Answers customers’ questions
Computes customers’ utility consumption
Provides estimates of bills to customers
Follows safety rules and procedures
Finds locations of meters

6. Job Analysis Data –KSAOs
Based on FDE
Ability to accurately read various types of electric and
water meters and accurately record the information -#1
Ability to listen to and comprehend oral instructions/
requests
Skill in reading maps/Ability to read maps and city street
directories
Skill in knowing how to find and use information
Knowledge of left, right, north, south, east, and west
Ability to read and follow written directions, operating
procedures, city policies, training bulletins, etc.
Knowledge of proper grammar and vocabulary
Ability to verify readings entered into processor
Skill in basic math computations

7. Written Test –Section One
Ability to accurately read various types of
electric and water meters -#1
Ability to accurately record information
from meter readings -#1
Meter Reading Accuracy
Presented via Power Point

8. Meter Reader Test for Meter Reader Test for
the City of Lakeland, FLthe City of Lakeland, FL
Assessment ServicesAssessment Services

9. Meter Reading Meter Reading
AccuracyAccuracy

10. ►►ExampleExample

11. ►►Get ready for the test items.Get ready for the test items.

12. 1.1.

13. 2.2.

14. Written Test –Section Two
Ability to listen to and comprehend oral
instructions/requests
Listening Comprehension
Actual Meter Reader Coordinators
Based on actual transmissions
Taped via radio for realistic effect
Allowed to take notes

15. Section 2Section 2
LISTENING LISTENING
COMPREHENSIONCOMPREHENSION

16. Scenario 1Scenario 1

17. 1.1.What needed to be rechecked What needed to be rechecked
by the meter reader?by the meter reader?

18. 1.1.What needed to be rechecked What needed to be rechecked
by the meter reader?by the meter reader?
A.A.The condition of the electric meterThe condition of the electric meter

19. A.A.The condition of the electric meterThe condition of the electric meter
B.B.The condition of the water meterThe condition of the water meter
1.1.What needed to be rechecked What needed to be rechecked
by the meter reader?by the meter reader?

20. A.A.The condition of the electric meterThe condition of the electric meter
B.B.The condition of the water meterThe condition of the water meter
C.C.The electric meter readingThe electric meter reading
1.1.What needed to be rechecked What needed to be rechecked
by the meter reader?by the meter reader?

21. A.A.The condition of the electric meterThe condition of the electric meter
B.B.The condition of the water meterThe condition of the water meter
C.C.The electric meter readingThe electric meter reading
D.D.The water meter reading The water meter reading
1.1.What needed to be rechecked What needed to be rechecked
by the meter reader?by the meter reader?

22. Written Test –Section Three
Skill in reading maps
Ability to read maps and city street
directories
Knowledge of left, right, north, south, east,
and west
Map Reading
Generic Map
Presented via PowerPoint on large screen

23. End of Listening Comprehension
Section

24. Map Reading Instructions

25. Section 3Section 3
Click to ContinueClick to Continue

27. Test Administrator: Please turn off the
projector.

28. Map Reading
You are at the corner of South 3rd Street
and Vine Street. Which statement is
correct?
A.South 5th Street is to the west.
B.South Railroad Street is to the east.
C.Cedar Street is to the north.
D.Chestnut Street is to the south.

29. Map Reading
You are traveling south on North 5th Avenue and
come to Valley Drive. In order to reach Chestnut
Street, you should:
A.turn right on Valley Drive and left on North
3rd Street.
B.turn left on Valley Drive and left on North
3rd Street.
C.turn right on Valley Drive and right on North
3rd Street.
D.turn left on Valley Drive and right on North
3rd Street.

30. Written Test –Part Four
Ability to read and follow written directions,
operating procedures, city policies, training
bulletins, procedure updates, equipment
manuals, etc.
Reading Comprehension

31. Reading Comprehension
Excerpts from Policy and Procedure
Manual:
-Common Problems Encountered (also a
realistic job preview)
-Getting Ready for the Day
-Estimating Meter Readings
-Meter Reading Processor
20 M/C questions

32. Written Test –Part Five
Knowledge of proper grammar and
vocabulary in order to communicate
effectively with others
English Usage
Vocabulary

33. English Usage and Vocabulary
15 M/C Questions –5 basic English
Usage, 10 Vocabulary
Chose vocabulary based upon common
terminology used within the department
Examples –“Defects,” “tampering,”
“status,” “precautions,” “obstacles”
Used in sentences with four answer
choices

34. Written Test –Part Six
Ability to verify readings entered into
processor
Checking

35. Checking
10 sets of four similar/alike number series
Have to identify how many are exactly
alike in each set
Example:
04012769
04412769
04017269
04012796

36. Written Test –Part Seven
Skill in basic math computations
Math Aptitude

37. Math Aptitude
Thegoalforameterreaderistohavenomorethanone(1)
estimatedreadingper100metersread.Ameterreaderreads
8,963metersinaone-monthperiodandreports92estimated
readings.Hasthegoalbeenachievedbythismeterreader?
A.Yes
B.No. The meter reader had approximately one too
many estimated readings.
C.No. The meter reader had approximately two too
many estimated readings.
D.No. The meter reader had approximately four too
many estimated readings.

38. Written Test –Part Eight
Ability to read maps and city street
directories
Skill in knowing how to find and use
information
Street Directory Usage
Applicants are provided a City of
Lakeland Street Directory

39. Street Directory Usage
Dyer MHP is located at:
A.415 Arizona Ave.
B. 11105 Old Dade City Rd.
C. 1525 Duff Rd.
D. 2302 Smithwood Dr.

40. Written Test –Part Nine
Ability to accurately read various types of
electric and water meters -#1
Ability to read and follow written
directions, operating procedures, etc.
Dial Comprehension
Examples with explanations are given in
test booklet

41. Meter Reading Accuracy
Instructions:Readeachofthefollowingfive-dialmetersusingthe
directionsbelowandchoosethebestanswer.Youaretofollowthese
instructionstoarriveatthecorrectanswer.Donotletanypreviously
acquiredcontradictorymeterreadingknowledgeaffectyouranswers.
1.Readeachdialofthefive-dialmeterfromlefttoright.
2.Whenthehandoneachdialfallsbetweentwonumbers,
thecorrectreadingisalwaysthesmallestnumber.
3.Whenthehandfallsbetweenthenumbers1and0,the
correctreadingisalways0.Inthiscase0standsfor“zero”.
4.Whenthehandfallsbetweenthenumbers9and0,the
correctreadingisalways9.Inthiscase0standsforthe
number10.

42. Dial Comprehension
A. 84328
B. 74218
C. 85329
D.74328
E. 75229

43. Piloting of Test
Newtestwasgivento8juniormeter
readers
Feedbackwassolicitedfromparticipants
(handout)
Testwasalsopilotedwithtwoindividuals
whodonotworkinthemeterreading
department–onerecentH.S.graduate,
onecollegestudent

44. Results of Test Pilot
.71 correlation between raw scores on
written test and accuracy rates
The higher the test score, the more
accurate meter reads were made

45. Realistic Job Preview
DesignedRJPtoinformapplicantsof
workenvironment
Presentedattestsessionpriortotest
Havehadapplicantschoosetoleave
WorkingonRJPvideowhichwillbe
shownatbeginningoftestsession
Listedonemploymentwebsitewhenjob
isposted.

46. Test Statistics
Have administered new test four times
Have hired ten employees
Have terminated one
Terminated individual did not score particularly
well -75.89% (25 applicants scored better, some
very significantly)
Was hired based on his previous experience
Did not do well on reading meters on test
Made many errors in the field
Others hired had much higher test scores –
90.69% avg. (Range 82.14% -99.11%)

47. More Statistics
Turnover rate for FY 2006 (Oct. 2005 –
present) –45% (9 employees)
Includes two promotions
One termination (new testing)
Two terminations (old testing)
Four resignations (old testing)
Turnover rate based on new testing –5%

48. Remaining Challenges
Complete RJP video
Continue monitoring item statistics
Continue monitoring job performance of those selected
Determine if any test section should be weighted
W ill get correlation between new em ployees’ accuracy
rates and test scores when they have enough time on
the job and we have enough new employees
Continue attempting to convince supervisors that job
analysis data should be used to develop interview
questions
Develop Field Test
Design creative and effective recruitment strategies

49. QUESTIONS
AND
ANSWERS