Presentation CIOMS VIII

CIOMS VIII signal management

1. CIOMS VIII – What to expect and
EMEA initiatives
Mr François MAIGNEN, PharmD, MSc (Paris),
MSc (London), GradStat
Principal Scientific Administrator, EMEA
(London)
http://www.linkedin.com/in/francoismaignen

2. Conflicts of interests (1)
Over the past 5 years, no direct or indirect conflicts of interest.
(a) No financial interests in the pharmaceutical industry
(b) No work carried out for the pharmaceutical industry whether or
not these activities have been subject to regular or occasional
remuneration in cash or kind.
(c) No other links with pharmaceutical industry e.g. grants for study
or research allocated by the industry, no fellowships or
sponsorships endowed by pharmaceutical industry e.g. IT
industry, manufacturing company, Directorship of recruitment
agency.
Public declaration of interest available from the Agency.

3. Disclaimer (1)
The views and opinions expressed in the following PowerPoint slides are
those of the individual presenter and should not be attributed to Drug
Information Association, Inc. (“DIA”), its directors, officers, employees,
volunteers, members, chapters, councils, Special Interest Area
Communities or affiliates, or any organization with which the presenter is
employed or affiliated.

These PowerPoint slides are the intellectual property of the individual
presenter and are protected under the copyright laws of the United
States of America and other countries. Used by permission. All rights
reserved. Drug Information Association, DIA and DIA logo are registered
trademarks or trademarks of Drug Information Association Inc. All other
trademarks are the property of their respective owners.

4. Disclaimer (2)
This presentation also includes some of
my views (my understanding) and not
necessarily those of the CIOMS VIII (or
any of its members).

5. Objectives of the presentation
•Understand the scope and objectives of CIOMS VIII
•Understand the fundamental concepts and
terminology of signal detection
•Understand the fundamental principles of the
quantitative methods, their advantages but also their
limitations
•This knowledge helps to have a rationale approach to
signal detection: integrating the quantitative methods
with the traditional methods of PhV
•Incl. important points concerning decision making
•Key EMEA initiatives

6. Increasing complexity
Increasing volume of
information (more signals)
PhV databases more and
more complex
Electronic reporting
New standards
ICH E2B(R), M2,
MedDRA, M5, ISO …).
Limited resources
Quantitative methods
(PRR, BCPNN, MGPS,
other descriptive
methods).
Signals
Resources /
Performances

7. •Added to the complexity
–Scientific challenges and
elements of interpretation
•Tightly linked to the (other)
pharmacovigilance activities
•These changes have revealed
some strength but also
deficiencies of our current
systems
–Lack of standardisation of
information
–Lack of international standards
for some critical terminologies
(medicinal products, active
substances, …)
–Data quality issues (incl. level
of documentation of the
reports, presence and handling
of duplicates, …)
–Handling of different languages

8. Dazed & confused?
Extensive literature
Methods?
Validation?
Performances?
Place in the
pharmacovigilance
artillery ?
Traditional vs
quantitative
methods?

9. Lost?

10. The need for guidance
Some (regulatory) guidance has been published on signal
detection activities, concerning the use of quantitative methods
and extensive scientific literature
Not internationally recognised, database specific
–Guidance for Industry – Good pharmacovigilance practices and
pharmacoepidemiologic assessment Food and Drug Administration
March 2005
–White paper FDA-PhRMA on the PhV data mining (Drug Safety
2005)
–Guideline on the use of statistical methods in EV-DAS
There is a need to have some clear, consensual and
internationally agreed guidance on the elements to consider
when doing signal detection

11. CIOMS VIII
Practical Aspects of Signal detection in
Pharmacovigilance
Between 20-30 members: world-wide regulatory
authorities (US, EU, JP, AU, CA), WHO, industry,
software providers, Experts (non-affiliated).
6 meetings: London, Geneva, WDC, Bonn, Paris.
Dr June Raine (MHRA) Chief of the EB.
The CIOMS VIII process is still ongoing and should
be finished by 1Q2009.

12. Signal
detection: a
complex
multifactorial
process

13. 1. Data
collection
Data capture
(management)
Data
transmission

14. 2. Methods,
signal
detection &
data analysis

15. 3. Signal
management
(prioritisation,
evaluation,
decision &
communication)

16. 4. Link with risk
management

17. 5. Quality
assurance &
points to consider

18. There are many points to consider
when doing signal detection …
The CIOMS is focusing on some of them:
Different methods? What are the
performances?
Concepts and terminology
Practical implementation of these
methods
Interpretation of the results
Following activities

19. 1. Data capture and data
management

20. Data capture and data
management (1)
Fundamental but not in the scope of CIOMS VIII
IT infrastructure and software
The volume of information hence the data
management activities (data coding, entry, recoding,
data quality) is extremely resource demanding.
Data management will have a critical influence on the
signal detection activities incl.
–Medicinal product information: creation and maintenance of
dictionaries, lack of international standard, absence of INN
or standards in some instances e.g. vaccines
–Medical terminology: criteria for the use of terms, conversion
of legacy data encoded with a different terminology, …
–Data quality: FUp, duplicates

21. 2. Methods: “traditional” and
quantitative methods

22. “Traditional” methods
Qualitative methods (opposed to quantitative)
Numerous data sources (SRS, RCTs, PASS,
etc ..)
Mostly empirical based on more or less
standardised scientific criteria (e.g. evaluation
of case reports - Sir Bradford-Hill criteria)
Not validated? CURRENT STANDARD

23. Quantitative methods
Primarily designed for SRS databases: Disproportionality analysis.
Drug-event pairs highlighted by the methods indicate a
disproportionality of reporting (increased observed value compared to
the expected value computed on the database, the drug-event pair is
on average reported more often than the majority of the drug-event
pairs in the database).
The underlying principle of this method is that a drug –event pair is
reported more often than expected relative to an independence model,
based on the frequency of ICSRs on the reported drug and the
frequency of ICSRs of a specific adverse event.
The drug-event pairs highlighted by the different data mining algorithms
are based on arbitrary thresholds and / or case count numbers defined
by the user.
For a same drug-event pair, the result of a DMA is DATABASE and
TIME specific.
One can find almost everything if the analysis is conducted on different
sets of data in the same database.

24. Advantages of the quantitative
methods
Operational viewpoint: allow a systematic screening
of the database to identify new safety issues (better
allocation of resources)
Fairly easy to implement (for some of them)
Public Health viewpoint:
–Can identify some signals (not all) before “traditional”
methods of PhV (the reverse is also true)
–The DMAs can identify signals which have not been picked
up by other methods (recent example is abacavir and
didanosine and risk of myocardial infarction see Lancet 371:
1417 and 372: 805).
–Quantitative methods have not been sufficiently evaluated
on a Public Health perspective.

25. Limitations of the quantitative
methods
The concept of threshold implies that not all the
reports will be reviewed and the quantitative methods
will not detect all the signals (for which the data have
been reported to the database on which the DMA is
used)
See Importance of reporting negative findings in data
mining – the example of exenatide and pancreatitis
Pharm Med 2008; 22(4): 215-219).

26. Limitations of the quantitative
methods
Important to identify the information
which must be reviewed systematically
(without relying on the use of the
quantitative methods)
–TMEs: Targeted Medical Events (link with
Risk Management)
–DMEs: Designated Medical Events
–Important Medical Events / Serious
Medical Events.

27. Limitations of the quantitative
methods
The disproportionality analysis is not an inferential
exercise (i.e. the method is not aimed at drawing
conclusions about a parent population on the basis of
evidence obtained from a random sample from this
population).
Limitation of the hypothesis testing.
This “statistical associations” detected by the
quantitative methods do not imply any kind of
causal relationship between the administration of
the drug and the occurrence of the adverse event.

28. Elements to consider when
integrating the two methods
Maximise the benefits of the quantitative
methods
Take into consideration their limitations.
The implementation of these methods is
highly situation dependent (volume of reports
and available resources, type of products,
type of database, resources available, trade-
off between true and false positive, number of
false negative, etc …).

31. 3. Signal detection, selection,
evaluation and decision making

32. Importance of interpretation
Need to define precisely the primary objective of the
search
When interpreting the results of DMAs it is important
to bear in mind …
–The limitations of the method of collection of data
(spontaneous reporting)
–The background chosen to perform the analysis (e.g.
vaccines, EudraVigilance vs AERS / Vigibase)
–The level / medical terms used to perform the analysis (e.g.
PT osteonecrosis, LLT: osteonecrosis of jaw)
–The criteria used to identify the suspected medication
(medicinal product / active substance) (e.g. Methotrexate).
–Use its medical judgement (which can hardly be
standardised in a guidance document)

33. Important concepts and
corresponding terminology
Different concepts, different
acronyms a great deal of
confusion
Signal of disproportionate
reporting (SDR), Signal and
Risk (established /
theoretical).
Many discussions AND no
real consensus on the
subject …

34. Different concepts / different
definitions
SDR (signal of disproportionate reporting): refer to drug-event
pairs highlighted by DMAs. (see EMEA guideline) NOTE: The
term SIGNAL in SDR will not be retained by the CIOMS VIII.
Signal: A signal is information on an adverse event that is new
or incompletely documented that may have causal relationship
to treatment and is recognized as being worthy of further
explorations (see CIOMS VIII). The SDRs must be
systematically medically confirmed.
(Identified) Risk: An untoward occurrence for which there is
adequate evidence of an association with the medicinal product
of interest (see Guideline on risk management systems for
medicinal products for human use EMEA/CHMP/96268/2005).

35. These concepts
connect directly
with the decision
making process

36. DMA
Database (drug-events pairs)
SDRs
SIGNALS
SIGNALS
(other data sources)
Medical judgement
RISKS
Further evaluation / characterisation
Regulatory
action
NO

37. I bet that the CIOMS IX, X, XI and
LXI will come up with a new
(different) definition of a signal …

38. And there will be no
consensus (again) …

39. The need for systematic
evaluation / characterisation
The upgrade of some SDRs to the status of
signals requires a medical judgement /
evaluation.
The SDRs need to be medically confirmed.
The signals require further evaluation /
characterisation.
No regulatory actions should be based only
on the basis of the presence of a SDR.

40. Process flow
included in the EMEA
guideline on the use
of statistical methods
implemented in the
EV data analysis
system
(EMEA/106464/06)
July 2008.

41. Signal management
Similarly the CIOMS has identified an additional
step which includes:
•Triage
•Prioritisation and impact analysis (see
presentation from Philippe)
•Evaluation
•Decision
•Communication (broad sense)
•Follow-up
•Link with risk management

42. 5. EMEA initiatives (and future
directions)

43. EMEA initiatives (1)
EudraVigilance and signal detection
–Revised EMEA guideline on the use of statistical
signal detection published in July.
–Finalisation of a performance study of the
quantitative methods implemented in EV-DAS.
–New Interface between EU-RMP and
EudraVigilance (EU-RMP Annex 1) (VB interface).
–New approaches to signal detection.

44. Stratification
New development and functionalities to
support signal detection activities scheduled
in 2009 (adjustment for confounding and sub-
grouping) (see latest Drug Safety)

45. EMEA initiatives (2)
EudraVigilance
–Publication of the access policies and start
of the technical implementation.
–Data management strengthened.
–Involvement in the international
harmonisation activities (ICH) and
standards development organisations
(ISO, CEN, HL7).

46. EMEA initiatives (3)
Pharmacoepidemiology, active
surveillance and R/B of medicines (incl.
communication).
–ENCEPP
–IMI Call No 6 – PROTECT (strengthening
the benefit / risk of medicines in the EU)

47. 6. Concluding remarks

48. Before we can fly …

49. Key points
Importance of concepts and terminology
Understand the methods, their strengths but
also their limitations – optimum integration
Define the primary objectives of the search
and take the time to interpret the results (i.e.
don’t rush)
The guidance will clarify some important
issues

51. [email protected]