Pitch detection in Tabla accompaniment

The slides for my MTech Thesis final presentation.

1. CSE, Indian Institute of Technology Bombay
Pitch Detection of Singing Voice in Tabla
Accompaniment
Ashutosh Bapat
(03305010)

2. CSE, Indian Institute of Technology Bombay
Outline
Motivation
Music transcription
Pitch & pitch detection
Signal characteristics
Two-way mismatch procedure
Post processing
DP based smoothing
Pitch correction
Experimental evaluation
Conclusion

3. CSE, Indian Institute of Technology Bombay
Automatic Music Transcription (AMT) system
Converts acoustic musical signal to symbolic
representation
Documents musical attributes
Pitch
Timbre
Rhythm
Pitch is the most salient
Melody = pitch contour

4. CSE, Indian Institute of Technology Bombay
AMT for Indian classical music
Components of Indian classical and semi-classical
music
Melody line sung by a single main voice
oGamakas: are described by detailed pitch contour
Accompaniment of tabla and tanpura
Musicological and pedagogical applications
Rich archives of audio recordings => need for a
reliable PDA for singing voice pitch tracking in tabla
accompaniment

5. CSE, Indian Institute of Technology Bombay
Pitch
Many definitions exist
Pitch of a signal is defined as the fundamental
frequency of an approximately harmonic pattern in
spectral representation of signal.
Pitch period is defined as average length of several
periods of the signal.
0
0
1
T
F=

6. CSE, Indian Institute of Technology Bombay
Pitch detection
•Input: musical signal
•Output: pitch contour
Song waveform Pitch contour

7. CSE, Indian Institute of Technology Bombay
Pitch Detection Algorithm
Preprocessor: Data reduction and enhancement
Commonly used method: Filtering
Basic Extractor: Estimates single/multiple pitch
candidates per frame
Commonly used method: ACF
Post processing:
Measure of reliability of each candidate
Smoothness of pitch contour
oCommonly used method: Dynamic programming

8. CSE, Indian Institute of Technology Bombay
Singing voice
•Pitch evolves
continuously
•Shows inflexions
like bents, stresses,
oscillations etc.

9. CSE, Indian Institute of Technology Bombay
Classification of tabla strokes
Number of drums
Simple strokes: Na, Ge, Ke, Tit, Tun etc.
Complex strokes: Dha, Dhin, Dhun
Harmonicity
Harmonic: Na, Tin, Tun, Ge
Inharmonic: Ke, Tit
Rate of Decay:
Slowly decaying: Na, Ge, Tin, Tun
Fast decaying: Ke, Tit

10. CSE, Indian Institute of Technology Bombay
Harmonic interference: Na

11. CSE, Indian Institute of Technology Bombay
Single partial interference: Ge

12. CSE, Indian Institute of Technology Bombay
Noisy interference: Ke

13. CSE, Indian Institute of Technology Bombay
Pitch detection of mixed song
Waveform of song mixed with
stroke Na
Pitch contour by ACF

14. CSE, Indian Institute of Technology Bombay
Two-way mismatch procedure
TWM error, F
0
= 300 Hz

15. CSE, Indian Institute of Technology Bombay
TWM and ACF: harmonic interference
•Complex tone of 450 Hz + signal
simulating Na
•In TWM error we can see
minimum at correct pitch
•In ACF all peaks are at lags
corresponding to 790 Hz
TWM error ACF
Magnitude plot

16. CSE, Indian Institute of Technology Bombay
TWM and ACF: single partial interference
•Complex tone of 300 Hz mixed with a single partial with
amplitude varied from 0 to 100.
•TWM is more robust than ACF
400 Hz 450 Hz

17. CSE, Indian Institute of Technology Bombay
TWM and ACF correlograms
•Correlograms of complex tone of 300 Hz mixed with stroke
Na
•Notice horizontal line at 300 Hz in TWM
•No clue to lag 73 (corresponds to 300 Hz)
ACF correlogram TWM error correlogram

18. CSE, Indian Institute of Technology Bombay
TWM pitch contour
•Pitch contour of song mixed with stroke Na
•Notice large pitch artifacts during strokes

19. CSE, Indian Institute of Technology Bombay
Post processing

20. CSE, Indian Institute of Technology Bombay
DP based smoothing
Smoothing based on
Measure of reliability of pitch candidates
Smoothness of pitch contour
Measurement cost:
Smoothness cost:
Local transition cost:
Global transition cost:
å
=
-=
N
j
jjpjpTNpjppS
1
)),1(),(())(,),(,),1(( 
))(),1(()),(()),1(),(( jpjpWjjpEjjpjpT -+=-
),(jpE
)',(ppW

21. CSE, Indian Institute of Technology Bombay
Smoothness cost
•The width of bell varies
proportional to pitch
•Pitch variation at high
pitches is expected to be
more than that at low
pitches
•Saturates at high values
pc
s
pp
ppW
e
*
)'(
1)',(
2
2
=
-
-=
-
s
s

22. CSE, Indian Institute of Technology Bombay
Pitch contour after applying DP
•Smoothened pitch contour
•Suppresses fast pitch variations
•May introduce errors where tabla is absent

23. CSE, Indian Institute of Technology Bombay
Pitch correction
•Searches for deepest local minimum in 6% range near pitch
estimated by DP
•Corrects most of the fine errors

24. CSE, Indian Institute of Technology Bombay
Experimental evaluation
Test samples
Samples produced by digitally adding tabla strokes Na,
Ge, Ke to pure song waveforms sung with syllable /la/
and /aa/
Algorithms
TWM:
TDP: TWM + DP
TDC: TWM + DP + PC
Errors
Fine error: error magnitude between 3% to 6%
Gross error: error magnitue above 6%

25. CSE, Indian Institute of Technology Bombay
Results
•DP has decreased number of gross errors increasing number
of fine errors
•PC has decreased number of fine errors
•Better performance in case of songs with slowly varying pitch
contours
TWM TDP TDC
FG FG FG
Na0.049.
3
4.613.
4
2.114.
8
Ge0.020.
9
3.92.13.42.5
Ke0.025.
7
4.95.10.25.1
TWM TDP TDC
FG FG FG
Na4.714.
7
11.
6
1.65.32.1
Ge0.022.
5
8.14.41.54.1
Ke0.017.
9
7.22.50.02.5
Song with many fast variations of pitchSong with slowly varying pitch contour
Error rates in percentage

26. CSE, Indian Institute of Technology Bombay
Errors after application of DP + PC
•Errors remaining after application of DP and pitch correction
are found in regions with fast variations in pitch

27. CSE, Indian Institute of Technology Bombay
Conclusion
Importance of music transcription
Characteristics of tabla strokes
Two-way mismatch PDA
Results showing improvements by application of DP
smoothing and pitch correction
Applications in building pitch detector for Indian
classical and semi classical music

28. CSE, Indian Institute of Technology Bombay
Future work
Combination of ACF and TWM to take advantage of
Lesser computational complexity of ACF
ACF’s robustness to noise, thus better results in Ke
Classification of frames by presence/ absence of
tabla strokes
Use pitch estimated by DP and pitch correction only in
frames containing tabla stroke
Application of advanced techniques:
adaptive windowing, peak selection, selective search
Pitch tracking in case of complex strokes like Dha
and words like TiReKiTa

29. CSE, Indian Institute of Technology Bombay
Thank you

30. CSE, Indian Institute of Technology Bombay
State space formulation of DP