RFP Evaluation Training

1. Request for Proposals (RFP)
Evaluation Training
Aviation Procurement Unit
March 26, 2009

2. 2
AGENDA
ƒWhen to Use an RFP Approach?
ƒOverview of the Evaluation Process?
ƒWhat are the Expectations for Evaluation Committee Members?
ƒWhat is the Approach to the Evaluation of Proposals and Scoring Guidelines?
ƒWhat is the Approach to Evaluating Technical Proposals?
ƒWhat is the Approach to Evaluating Cost Proposals?
ƒOverview and Steps of Consensus Scoring?
ƒBest Practices for Evaluators
ƒNext Steps
ƒQuestions and Answers

3. 3
Best Value Should Always be the
Purpose of the Evaluation Process?
A Best Value
Award is one
which optimizes quality,
performance, cost, time, and
efficiency in a fair manner to
the contractor community

4. 4
When is an RFP the Right Approach?
ƒThe Bid process is the default solicitation option.
ƒWhen we are not sure of the best solution and multiple options
exist.
ƒWhen performance based incentive contracts are used.
ƒWhen complex factors in addition to price are very important.
ƒWhen there are complex requirements.
ƒWhen it is a new program/ proj ect that has not been tested.
ƒWhen it is desirable to find out if there are alternative options to
what is normally performed.

5. 5
What is the Best Way to Handle
Criteria for RFP Evaluations?
ƒAll evaluation criteria MUST be established prior to the
issuance of RFP.
ƒCriteria should be clearly linked to desirable outcomes.
ƒBe clear and concise in language to avoid confusion
ƒEvaluation information should be documented, dated,
and secured throughout the evaluation process.

6. 6
How Do We Develop an Evaluation
Strategy? ƒIdentify (eliminating) Minimum Qualifications
ƒDefine Criteria
ƒDevelop Specifications
ƒDecide importance of Oral Interviews/ Presentations
ƒIncorporate Reference Checks into Industry Experience
ƒDecide importance of Site Visits
ƒHow important is Cost; Cost to Technical Score Ratio?
ƒDetermine distribution of points among each criteria

7. 7
What are Examples of Standard
Criteria in a Technical Proposal?
ƒCorporate Experience:
ƒWhat are the proponents’ past performance and experience in
similar types of contracts?
ƒWhat is the experience level of the essential subcontractors?
ƒProject Organization and Staffing:
ƒWhat is the staffing approach?
ƒWhat are the qualifications and past performance of the personnel?
ƒImplementation Approach:
ƒWhat is the adequacy of their work breakdown structure/ project
management?
ƒIs it Comprehensive? Reasonable?
ƒProposed Deliverables?
ƒPreparation Plan?

8. 8
What are Examples of Standard
Criteria in a Technical Proposal?
ƒOperation/ Management Approach:
ƒPlan for Regular/ Daily Operations?
ƒApproach to Quality Control and Assurance?
ƒWhat are the Commitments to Performance Standards?
ƒWhat are the Plans for Maintenance?
ƒWhat are the Contingency procedures in the event of an emergency?
ƒCorporate and General Capacity:
ƒWhat is the overall capacity of the organization?
ƒDo they have a plan to sufficiently staff the contract project?
ƒHow long has the company been operating?
ƒWhat is the turnover ratio?

9. 9
Overview of the Evaluation Process
ƒRules and regulations that are applicable to the
evaluation process should be followed throughout the
entire process.
ƒDOP makes a determination of responsiveness and
responsibility based upon minimum qualifications.
ƒAPU facilitates and coordinates process between DOP
and evaluators

10. Overview of the Evaluation Process
ƒEvaluation committee members independently
review proposals based upon the specific
evaluation criteria
ƒTechnical and cost scores are assigned
applying the designated evaluation criteria
ƒOral interviews are conducted, if necessary
ƒNegotiations are conducted with the winning
proponent
10

11. 11
What are Standards of Conduct While
Serving as an Evaluator?
ƒNo Conflicts of Interest
ƒNo solicitation or acceptance of gifts or anything
of value
ƒIntegrity and Honesty during scoring and
communications
ƒProper handling and discretion of all information;
No Disclosure during process
ƒPractice Ethical Behavior

12. 12
What is Expected from
Evaluation Committee Members?
ƒThe member has knowledge or experience regarding the RFP Subject
Matter
ƒThe member has sufficient time to dedicate in order to effectively
evaluate the proposals.
ƒSuch time is dedicated to reading all proposal related materials
extensively
ƒAttending all Evaluation-related Meetings: Training, Evaluation
Sessions, Oral Interviews, Debriefings, Criteria Planning
ƒConduct an independent review.
ƒIdentify potential deficiencies in a proposal that could result in a
determination of non-responsiveness or non-responsibility.
ƒCommunicate any issues to the Contracting Officer immediately.
ƒLimit communications regarding the RFP to those DOA and DOP staff
who are directly assigned to the project.

13. 13
What is the Purpose of Oral
Interviews?
ƒAllows proponents to clarify on areas
that may be unclear in their proposals
ƒPossibly reduces the likelihood of
misunderstandings.
Evaluation committee members determine whether or not oral interviews will be necessary.
•Oral interviews should follow group discussions
•Oral interviews should ALWAYS
precede scoring.

14. 14
What is the Purpose of Reference
Checks?
ƒReference Checks may be conducted with prior customers of the
proponent
ƒReference Checks may also include references of key personnel.
ƒExamples of Questions include:
ƒWere the projects/ deliverables completed on time?
ƒWere there any problems with the management team?
ƒWere the Project Management plans realistic?
ƒWere there any other significant problems?
ƒDid the Prime Contact stay the same or change over the duration of the
project?
ƒWere the Personnel Knowledgeable and Competent?
ƒWould you do business with this vendor/ contractor again?
ƒHow would you rate your level of satisfaction from 1-5?
Results of Reference Checks will be distributed to the evaluation team prior to scoring

15. Ensuring Security and Confidentiality
ƒKeep proposals in a secure place.
ƒEvaluators shall not discuss the co ntents of submitted proposals
or the procurement activities with any persons outside of the
evaluation team or Procurement staff.
ƒEvaluators shall not communicate the scoring outcomes or
content of proposals and shall not disclose the status of any
proposal.
15

16. 16
How are Proposals Evaluated?
All Proposals shall be evaluated in
accordance with the City’s Code of
Ordinances and the criteria specified
on the Percentage Evaluation Form
and considering the information
required to be submitted in each
Proposal.

17. 17
Evaluation Scoring Guidelines
ƒScoring will be conducted for each criterion on a scale of 1-10
ƒTechnical Criteria will be scored as such:
ƒ8-10:Oustanding: Proponent’s proposal exceeds expectation and
demonstrates an excellent ability to reach the goals and objectives of
the procurement. Proposal is consistently strong.
ƒ5-7: Highly Effective: Proponent’s proposal demonstrates a good
ability to reach the goals and objectives of the procurement.
ƒ3-4: Effective: Proponent’s proposal demonstrates a fair ability to
reach the goals and objectives of the procurement.
ƒ1-2: Marginally Effective: Prop onent’s proposal demonstrates a
minimal ability to reach the goals and objectives of the procurement.

18. 18
Evaluation Scoring Guidelines
ƒScores will be adjusted to reflect the weight assigned
to each criterion.
ƒEach criterion must be reviewed according to the
requested items of the RFP and scored based on the
scoring definitions above.
ƒOnly the Criteria in the Evaluation Form will
receive scores.
ƒAny Proponents Proposal that does not meet the
minimum requirements of the RFP should be identified
as this may deem them non-responsive.

19. 19
When is a Proposal Deemed
Non-Responsive?
ƒOnly the Department of Procurement has authority to deem a
proponent non-responsible and/or non-responsive.
ƒEvaluators should notify the DOP Contracting Officer of
deficiencies and/or clarifications needed.
ƒAll minimum requirements will be scored on a pass/fail basis.
ƒFailure to pass all mandatory requirements may deem a proposal
nonresponsive
ƒe.g., number of similarly completed projects
ƒEvaluators should promptly notify Procurement staff if there is a
problem with the proposal.

20. What Approach Should We Use to
Evaluate the Technical Proposal?
ƒMinimum requirements should be evaluated on
a pass/fail basis.
ƒReview the established criteria and related
weights.
ƒEvaluate proposals strictly by DOA’scriteria as
defined in the RFP for each proposal.

21. 21
What Should Technical Scoring Look
Like?
Weight
Criteria
Company 1
Company 2
Company 3
20%
Organization/Resumes of
Key Personnel
10
3
1
30%
Overall Experience, Qualifications and Performance of
Previous Similar Projects
9
7
4
20%
Strategic Approach
To Increasing
Federal Assistance to the
City through Government
Agencies and Congress
8
6
3
ƒOnly the criteria indicated will receive scores.
ƒScoring will be conducted on a scale of 1-10
ƒTechnical Scoring should reflect the needs of the Proposal

22. 22
What Approach Should We Use to
Evaluate the Cost Proposal?
ƒAfter technical aspects of prop osal responses are initially
reviewed, then cost proposals are reviewed.
ƒCost proposals may reflect price escalation, deceleration due to
performance, time, CPI, etc. may be applied.
ƒCost Evaluation Must be independent of the Technical Evaluation.
ƒA Clear Formula should be used to Calculate Cost Scores.
ƒScores should be directly linked solely to costs proposed on a
formulaic basis.
ƒCost proposals/ financial offers mu st be taken at face value when
the cost proposal is evaluated.

23. Examples of Cost Proposals…
Performance Based Contract
RFPs may read like this…..
Please propose Base Costs
and Incentives associated
with performance beyond
mandatory requirements…
E.g., Beyond 90% Customer
Satisfaction Scores
consecutively for one year will
lead to a 3% bonus (e.g..,
janitorial)
Cost Savings Contract RFPs may read like this…..
Please propose Base Costs and
Incentives associated with cost
savings beyond mandatory
requirements…
E.g., Company X will receive
10% of all cost savings beyond
performance guarantee (e.g..,
workers’ compensation)
23

24. Let’s Take a Look at the Cost Proposal
Scoring…
ƒThe lowest fee offer receives the maximum points possible (10). In the event,
that it is a revenue generating prop osal, it would be the highest offer.
ƒPoints are distributed based upon percent ile in which the fee offer is found in
relationship to the lowest offeror.
ƒ
Formula Used: Lowest Offer/Offer at hand= Percentage Relationship to Lowest Offer
24
EXAMPLE
Cost Offer
Points
Cost Proposal 1
$780k
10 points
Lowest Offer
Cost Proposal 2
$1.3 million
6 points
780K/1.3 million= .6
Cost Proposal 3
1.8 million
4.3 points
780k/ 1.8 million= .43

25. 25
Let’s Recap all Scoring Elements…
Percentage Weights
Criteria
10
Corporate Experience
15
Project Organization and Staffing
15
Implementation Approach
20
Operation/ Management Approach
10
Corporate and General Capacity
10
Cost/ Financial Offer
15/0
OCC (Minority Participation)
5
Financial Capacity
100
TOTAL

26. 26
How Does Consensus Scoring Work?
Consensus Scoring consists of the
Entire Evaluation Committee
participating in Joint Discussion,
Assessment, and Scoring of
Technical and Cost Criteria in order to
develop one unified score for each
criteria.

27. 27
ƒTeam discussion leads to greater sharing of each
other’s Expertise in cross- functional projects, which
are quite common in DOA RFPs
ƒGenerates clarification where misunderstandings may
Exist
ƒEnsures that all evaluators are truly engaged in the
Evaluation process
Let’s Review the Benefits of
Consensus Evaluation and Scoring…

28. Let’s Review the Benefits of
Consensus Evaluation and Scoring…
ƒEncourages deeper probing, questioning, discussion,
and analysis of proposals to the evaluation criteria
ƒA unified understanding of responses to all proposals
is reached
ƒEliminates discrepancies among scores
ƒOverall, the result is a more comprehensive and
accurate assessment of the proposals
28

29. 29
How Should Individual Evaluators Prepare Prior to
the Consensus Evaluation Session?
ƒReview thoroughly and comprehensively all proposal responses,
repeating review of Responses where necessary.
ƒKeep in mind how the information relates specifically to the stated
criteria in the RFP.
ƒIdentify all points of informat ion in the responses that need
clarification
ƒIdentify the pros and benefits of each proposal response.
ƒIdentify the drawbacks, cons, and concerns of each proposal
Response.
ƒEvaluators should not
assign scores during independent review of
proposal.

30. 30
What are the Steps to the Consensus
Scoring Evaluation Session?
ƒThe Evaluation team continues until al l desired elements of discussion are
exhausted by all evaluators.
ƒEach Member indicates points of information where they need clarification and ask
questions to fill such voids, when possibl e, with other evaluation team members.
ƒEach Member identifies the Pros and Benefits of each proposal.
ƒEach Member identifies the Cons, Costs, and Concerns of each proposal.
ƒEach Member communicates their assessment of the Overall Quality of each
Proposal Response based upon each criteria.
ƒOnly after all Proposals are discussed, then each proposal response is scored by
the team based upon all criteria.
ƒThe Facilitator records the Final scor es for Documentation and Reporting.

31. 31
What is the Role of the Facilitator in
the Consensus Evaluation Session?
ƒProvide Structure and Ensure that all Evaluators Participate and
Contribute to the Process in a Balanced and Generous manner
ƒMonitor and Ensure that an Objective Stance is upheld
throughout the entire Evaluation Process.
ƒKeep the Team Focused and Ensure that Each Criteria is
Discussed by the Entire Team
ƒRemind Evaluators to Focus Strictly on the Evaluation Criteria
ƒEnsure that all Needed Clarification is Obtained
ƒCalculate Final Scores and Rankings

32. Important Documentation Steps for
Consensus RFP Evaluations…
ƒTechnical Evaluation matrix (one score for each company)
ƒCost Evaluation Scores (one score for each company)
ƒTotal Final Score Matrix
ƒEvaluation Session Notes by Facilitator IMPORTANT : The Facilitator/ Procurement Staff takes notes during
the Collaborative Evaluation Session.
32

33. 33
Best Practices in Evaluations
ƒMake every effort to differentiate the quality among proposals reviewed.
ƒRFP Evaluations for each contract are expected to be complete within 10
days.
ƒKeep in mind that you are making a decision that has an impact on
significant dollars, often millio ns- be dedicated and focused.
ƒCost proposals should be evaluated strictly based on a formula.
ƒTechnical proposals should be evaluated strictly by the already developed
criteria.
ƒBenchmark costs to ensure reasonableness of proposed costs.
ƒThoroughly read the evaluation criteria.
ƒAttend every scheduled evaluation team meeting and contribute to the
discussion.

34. 34
Now the Recommendation has to be
approved…
ƒThe Project Manager and respective AGM writes a
supporting letter for the recommendation by the
Evaluation Committee.
ƒThe General Manager sponsors the recommendation
and sends to the Chief Procurement Officer.
ƒCity Council and mayor must approve the
recommendation through legislation in order for the
City to enter into the contract with the proponent.
ƒThe Contract must be signed by the GM, Law, the
CPO, the Mayor and the Clerk.

35. 35
Questions & Answers
Aviation Contracts Area
Lepora Manigault, Procurement Officer – 404.530.2421
Justina Mann, Contracts Supervisor – 404.530.5408
Diane Akinsipe, Contracting Officer – 404.209.2922
Erin Jenkins, Contracting Officer – 404.209.3171