Embedded Recipes 2019 - Herd your socs become a matchmaker

About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing.

In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra

Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer.
Geert Uytterhoeven

1. Herd Your Boards, Become a Farmer
Embedded Recipes 2019
Geert Uytterhoeven
[email protected]
GLIDER bvba
September 23, 2019
© Copyright 2019 GLIDER bvba, CC BY-SA 4.0

2. Herd Your SoCs, Become a Matchmaker
Embedded Recipes 2019
Geert Uytterhoeven
[email protected]
GLIDER bvba
September 23, 2019
© Copyright 2019 GLIDER bvba, CC BY-SA 4.0

3. Table of Contents
Introduction
History
Device Tree Bindings
Drivers
Final Words
2 / 35

4. My Linux Subsystems
IMaintainer of Renesas clock and pin control drivers
since 2016
IMaintainer of Renesas ARM SoC platforms
since July 2019
ILTSI Submaintainer
since July 2018
IMaintainer of the m68k architecture
since 2004
3 / 35

5. My Linux Subsystems
IMaintainer of Renesas clock and pin control drivers
since 2016
IMaintainer of Renesas ARM SoC platforms
since July 2019
ILTSI Submaintainer
since July 2018
IMaintainer of the m68k architecture
since 2004
3 / 35

6. What Is Matching?
Finding a suitable driver for a piece of hardware
+Identifying and using (all) available capabilitiesHandling hardware-specic quirks/bugs/limitations
4 / 35

7. What Is Matching?
Finding a suitable driver for a piece of hardware
+Identifying and using (all) available capabilitiesHandling hardware-specic quirks/bugs/limitations
4 / 35

8. What Is Matching?
Finding a suitable driver for a piece of hardware
+Identifying and using (all) available capabilitiesHandling hardware-specic quirks/bugs/limitations
4 / 35

9. Table of Contents
Introduction
History
Device Tree Bindings
Drivers
Final Words
5 / 35

10. A long time ago in a galaxy far, far away ...
x86
IKernel conguration: per-machine!not scalable!
IISA probing
Multi-platform
IMachine type (m68k, MIPS, SPARC)
ITagged bootinfo (m68k, ARM, MIPS)
IInfo passed by bootloader or IDPROM
6 / 35

11. A long time ago in a galaxy far, far away ...
x86
IKernel conguration: per-machine!not scalable!
IISA probing
Multi-platform
IMachine type (m68k, MIPS, SPARC)
ITagged bootinfo (m68k, ARM, MIPS)
IInfo passed by bootloader or IDPROM
6 / 35

12. Rise of the Discoverable Buses
Discoverable Buses
IPCI
IZorro (Amiga)
INuBus (Mac)
ISBUS (SPARC)
I. . .
find()API
Ipci_find_device()
Izorro_find_device()
Ifor_each_sbus{,dev}()
I. . .
I+ Resource management
request_{mem_}region()
Image © 20th Century Fox
7 / 35

13. Rise of the Discoverable Buses
Discoverable Buses
IPCI
IZorro (Amiga)
INuBus (Mac)
ISBUS (SPARC)
I. . .
find()API
Ipci_find_device()
Izorro_find_device()
Ifor_each_sbus{,dev}()
I. . .
I+ Resource management
request_{mem_}region()
Image © 20th Century Fox
7 / 35

14. Device Framework (2001)
ITriggered by hot-pluggable buses (e.g. USB)
ISeparation bus / device / driver
Ipci_{device,driver}
Izorro_{device,driver}
Iplatform_{device,driver}
I. . .
ICore matches devices to drivers based on IDs
match()API
structbus_type {
int(*match)(structdevice*dev,
structdevice_driver*drv);
};
8 / 35

15. Device Framework (2001)
ITriggered by hot-pluggable buses (e.g. USB)
ISeparation bus / device / driver
Ipci_{device,driver}
Izorro_{device,driver}
Iplatform_{device,driver}
I. . .
ICore matches devices to drivers based on IDs
match()API
structbus_type {
int(*match)(structdevice*dev,
structdevice_driver*drv);
};
8 / 35

16. Embedded: Return Of The Non-Discoverable Buses
Non-Discoverable Buses
IGrowing integration: SoC
IExponentially growing
number of SoC families,
versions, and revisions
IStandardized components on
board
IHardware description:
board leswith
platform_device+
platform_data
ILinus' 2010 rant about ARM
churn
Image © 20th Century Fox
9 / 35

17. Device Tree
IReal Open Firmware
IFlattened Device Tree
IDevice,device node
ICompatible value(s)
IStandard properties (resources, . . . )
ICustom properties
IPhandles
ISubnodes
IClear separation betweenHardware Descriptionand
OS Conguration
IStable DT ABI
ITies intoplatform_device
10 / 35

18. DT Early Adopters
IStraight conversion fromplatform_datato DT
properties
ISame compatible value, many feature properties
)All detail in DT
ICCF: Collection of device nodes for on-SoC clock
generator
IStandard and custom parts
IIndividual bits described in DT
Caveats
Youwillforget to describe a difference, difcult to x later!
INew SoC reuses block with e.g. different predivider
)DT property
INewer block with different predivider now also supported!
IWhat if another difference is discovered later?
11 / 35

19. DT Early Adopters
IStraight conversion fromplatform_datato DT
properties
ISame compatible value, many feature properties
)All detail in DT
ICCF: Collection of device nodes for on-SoC clock
generator
IStandard and custom parts
IIndividual bits described in DT
Caveats
Youwillforget to describe a difference, difcult to x later!
INew SoC reuses block with e.g. different predivider
)DT property
INewer block with different predivider now also supported!
IWhat if another difference is discovered later?
11 / 35

20. DT Evolution/Maturity
IMore different compatible values, few/no feature properties
IClocks: "monolithic" device node for on-SoC clock
generator
I)Easier to extend/maintain
ELCE2014:Engaging Device Trees
IStill good?
Yes! (surprisingly?)
IClocks:
IMoved from collection of nodes to a monolithic approach
I(DT without Common Clock Framework is dead)
IPower Domains: similar, but didn't have DT bindings yet
https://elinux.org/ELC_2014_Presentations
12 / 35

21. DT Evolution/Maturity
IMore different compatible values, few/no feature properties
IClocks: "monolithic" device node for on-SoC clock
generator
I)Easier to extend/maintain
ELCE2014:Engaging Device Trees
IStill good?
Yes! (surprisingly?)
IClocks:
IMoved from collection of nodes to a monolithic approach
I(DT without Common Clock Framework is dead)
IPower Domains: similar, but didn't have DT bindings yet
https://elinux.org/ELC_2014_Presentations
12 / 35

22. DT Evolution/Maturity
IMore different compatible values, few/no feature properties
IClocks: "monolithic" device node for on-SoC clock
generator
I)Easier to extend/maintain
ELCE2014:Engaging Device Trees
IStill good?
Yes! (surprisingly?)
IClocks:
IMoved from collection of nodes to a monolithic approach
I(DT without Common Clock Framework is dead)
IPower Domains: similar, but didn't have DT bindings yet
https://elinux.org/ELC_2014_Presentations
12 / 35

23. soc_device_match() (2016)
IDifferent approach, in-kernel
IWildcards!
IIf all else fails. . .
structsoc_device_attribute {
const *machine;
const *family;
const *revision;
const *serial_number;
const *soc_id;
const *data;
};
const soc_device_attribute *soc_device_match(
const soc_device_attribute *matches);
13 / 35

24. Table of Contents
Introduction
History
Device Tree Bindings
Drivers
Final Words
14 / 35

25. Designing Compatible Value Schemes
The Original Series
Basic Idea
IMultiple compatible values: most-specic to least-specic
IDriver matches against what is supported
Original: current version + older compatible version
ISuitable for slow evolution
E.g. 8250!16450!16550 UART Bad example!
IDiscrete components
E.g. WLAN controller
15 / 35

26. Designing Compatible Value Schemes
The Original Series
Basic Idea
IMultiple compatible values: most-specic to least-specic
IDriver matches against what is supported
Original: current version + older compatible version
ISuitable for slow evolution
E.g. 8250!16450!16550 UART Bad example!
IDiscrete components
E.g. WLAN controller
15 / 35

27. Designing Compatible Value Schemes
A Brave New World
Fast Evolution
IBuilding an SoCLEGO, Kcong
IComplex heritage hierarchy (copy and corrupt^Wmodify)
IWhich one is predecessor/successor?
IDo all family members have the same version?
SiFive
sifive,<ip-block-name><integer version number>
E.g.sifive,uart0
16 / 35

28. Designing Compatible Value Schemes
A Brave New World
Fast Evolution
IBuilding an SoCLEGO, Kcong
IComplex heritage hierarchy (copy and corrupt^Wmodify)
IWhich one is predecessor/successor?
IDo all family members have the same version?
SiFive
sifive,<ip-block-name><integer version number>
E.g.sifive,uart0
16 / 35

29. Example: Renesas DT Bindings For On-SoC Devices
1.SoC-specic(always, unless version register present)
E.g.renesas,r8a7791-sysc,renesas,r8a7795-wdt
!To anticipate unknown differences
2.Family-specic(usually)
E.g.renesas,rcar-gen2-scif
3.Generic(sometimes: legacy or version register)
E.g.renesas,sciforrenesas,vsp2
IDT bindings document all of the above (checkpatch.pl!)
IDTS advertizes all of the above
IDriver matches against least-specic that will do the job
17 / 35

30. Example: Renesas DT Bindings For On-SoC Devices
1.SoC-specic(always, unless version register present)
E.g.renesas,r8a7791-sysc,renesas,r8a7795-wdt
!To anticipate unknown differences
2.Family-specic(usually)
E.g.renesas,rcar-gen2-scif
3.Generic(sometimes: legacy or version register)
E.g.renesas,sciforrenesas,vsp2
IDT bindings document all of the above (checkpatch.pl!)
IDTS advertizes all of the above
IDriver matches against least-specic that will do the job
17 / 35

31. Renesas DT Compatible Values: Statistics
Linux v5.3:
TypeDocumentedUsedMatched
SoC-specic 743688 296
Family-specic 92 82 85
Generic 30 29 28
Is proliferation of compatible values, and churn on the DT
binding documents, worth it?
18 / 35

32. Renesas DT Compatible Values: Statistics
Linux v5.3:
TypeDocumentedUsedMatched
SoC-specic 743688 296
Family-specic 92 82 85
Generic 30 29 28
Is proliferation of compatible values, and churn on the DT
binding documents, worth it?
18 / 35

33. Example: Differences Within The Same Family
IObvious for e.g. clocks, power domains, pin controllers,
IUnexpected (R-Car Gen3 only):
DRIF
LVDS
PCIe
RPC-IF
Sound
Thermal Tj,
USBHS
E3/D3 use a different PLL,
VIN
ICompare:23family-specic values for R-Car Gen3.
19 / 35

34. Table of Contents
Introduction
History
Device Tree Bindings
Drivers
Final Words
20 / 35

35. Supporting a Variety of Similar Devices
Similar devices on (a family of) SoCs may have:
ISlightly differing feature sets
IQuirks/bugs
Differentiate by:
ICompatible value
IMachine, family, revision, serial number, soc_id
+ wildcards (e.g..revision = "ES1.*")
IDevice address?
Solutions
1.
2.struct soc_device_attribute and
soc_device_match()
21 / 35

36. Supporting a Variety of Similar Devices
Similar devices on (a family of) SoCs may have:
ISlightly differing feature sets
IQuirks/bugs
Differentiate by:
ICompatible value
IMachine, family, revision, serial number, soc_id
+ wildcards (e.g..revision = "ES1.*")
IDevice address?
Solutions
1.
2.struct soc_device_attribute and
soc_device_match()
21 / 35

37. Supporting a Variety of Similar Devices
Similar devices on (a family of) SoCs may have:
ISlightly differing feature sets
IQuirks/bugs
Differentiate by:
ICompatible value
IMachine, family, revision, serial number, soc_id
+ wildcards (e.g..revision = "ES1.*")
IDevice address?
Solutions
1.
2.struct soc_device_attribute and
soc_device_match()
21 / 35

38. Quirk Classes
Variant-Specic
IHow to handle? Device compatible value
IExample: New or missing features
SoC-Specic / SoC Integration Issue
IHow to handle?
IDevice compatible value
Isoc_device_match()
IExample: Broken DMA on early revision
22 / 35

39. Quirk Classes
Variant-Specic
IHow to handle? Device compatible value
IExample: New or missing features
SoC-Specic / SoC Integration Issue
IHow to handle?
IDevice compatible value
Isoc_device_match()
IExample: Broken DMA on early revision
22 / 35

40. Quirk Classes
Instance-Specic
One instance on SoC affected
IHow to handle?
IDevice base address +soc_device_match()
IDT property / Device compatible value
IExample: Broken interrupt on one instance
Board-Specic
IHow to handle? Board compatible value
IExample: R-Car Gen2 regulator quirk
23 / 35

41. Quirk Classes
Instance-Specic
One instance on SoC affected
IHow to handle?
IDevice base address +soc_device_match()
IDT property / Device compatible value
IExample: Broken interrupt on one instance
Board-Specic
IHow to handle? Board compatible value
IExample: R-Car Gen2 regulator quirk
23 / 35

42. Matching Methods
1.platform_driver.driver.of_match_table
2.*_OF_DECLARE()
XEarly (for clocks, timers, irqchips)
X
X -EPROBE_DEFER
IMixed:CLK_OF_DECLARE_DRIVER
3.for_each_compatible_node()
of_find_compatible_node()
of_find_matching_node()
of_device_is_compatible()
4.soc_device_match()
24 / 35

43. Matching Methods
1.platform_driver.driver.of_match_table
2.*_OF_DECLARE()
XEarly (for clocks, timers, irqchips)
X
X -EPROBE_DEFER
IMixed:CLK_OF_DECLARE_DRIVER
3.for_each_compatible_node()
of_find_compatible_node()
of_find_matching_node()
of_device_is_compatible()
4.soc_device_match()
24 / 35

44. Matching Methods
1.platform_driver.driver.of_match_table
2.*_OF_DECLARE()
XEarly (for clocks, timers, irqchips)
X
X -EPROBE_DEFER
IMixed:CLK_OF_DECLARE_DRIVER
3.for_each_compatible_node()
of_find_compatible_node()
of_find_matching_node()
of_device_is_compatible()
4.soc_device_match()
24 / 35

45. Matching Methods
1.platform_driver.driver.of_match_table
2.*_OF_DECLARE()
XEarly (for clocks, timers, irqchips)
X
X -EPROBE_DEFER
IMixed:CLK_OF_DECLARE_DRIVER
3.for_each_compatible_node()
of_find_compatible_node()
of_find_matching_node()
of_device_is_compatible()
4.soc_device_match()
24 / 35

46. Naive/Bad Examples
if(of_device_is_compatible(np,)) {
...
}
if(soc_device_match(&myrev1)
soc_device_match(&myrev2)) {
...
}
ICalled multiple times?
IFrom an interrupt handler?
INot scalable!
25 / 35

47. Naive/Bad Examples
if(of_device_is_compatible(np,)) {
...
}
if(soc_device_match(&myrev1)
soc_device_match(&myrev2)) {
...
}
ICalled multiple times?
IFrom an interrupt handler?
INot scalable!
25 / 35

48. Example: Scalar Feature Mask
drivers/clk/renesas/rcar-gen3-cpg.c
static soc_device_attribute cpg_quirks_match[]
{
.soc_id, .revision,
.data void*)(PLL_ERRATA
},
...
};
intrcar_gen3_cpg_init(...)
{
const soc_device_attribute *attr;
attr
if(attr)
cpg_quirks uintptr_t)attr->data;
}
// Later...
if(cpg_quirks
26 / 35

49. Example: Feature Structure, Combined
drivers/media/platform/rcar-vin/rcar-core.c
static of_device_id rvin_of_id_table[]
{
.compatible,
.datarcar_info_r8a7795,
}, ...
};
static soc_device_attribute r8a7795es1[]
{
.soc_id, .revision *",
.datarcar_info_r8a7795es1,
}, ...
};
static rcar_vin_probe( structplatform_device *pdev)
{
vin->info&pdev->dev);
attr
if(attr)
vin->info->data;
}
27 / 35

50. Different DTSes For Different SoC Revisions
IInclude new SoC.dtsiin old.dtsi
IAdd/override nodes and properties
I/delete-node/,/delete-property/
IUsesoc_device_match() to differentiate
Example
IR-Car H3 ES1.x (R8A77950) and ES2.0 (R8A77951)
both user8a7795-based compatible values
IIn hindsight:
IES2.0 is a new SoC, not just a xed ES1.x
INewr8a77951-based compatible values
28 / 35

51. Different DTSes For Different SoC Revisions
IInclude new SoC.dtsiin old.dtsi
IAdd/override nodes and properties
I/delete-node/,/delete-property/
IUsesoc_device_match() to differentiate
Example
IR-Car H3 ES1.x (R8A77950) and ES2.0 (R8A77951)
both user8a7795-based compatible values
IIn hindsight:
IES2.0 is a new SoC, not just a xed ES1.x
INewr8a77951-based compatible values
28 / 35

52. Table of Contents
Introduction
History
Device Tree Bindings
Drivers
Final Words
29 / 35

53. Has DT improved the ARM situation?
Evolution of the number of ARM defcong les (ELC2014) 0
20
40
60
80
100
120
140
160
180
v2.6.14
v2.6.19
v2.6.24
v2.6.29
v2.6.34
v2.6.39
v3.4
v3.9
v3.14
30 / 35

54. Has DT improved the ARM situation?
Evolution of the number of ARM defcong les 0
20
40
60
80
100
120
140
160
180
v2.6.14
v2.6.19
v2.6.24
v2.6.29
v2.6.34
v2.6.39
v3.4
v3.9
v3.14
v3.19
v4.4
v4.9
v4.14
v4.19
v5.3
31 / 35

55. Has DT improved the ARM situation?
Evolution of the number of ARM DTS les 0
200
400
600
800
1000
1200
v3.4
v3.9
v3.14
v3.19
v4.4
v4.9
v4.14
v4.19
v5.3
arm32
arm64
32 / 35

56. Has DT improved the ARM situation?
Evolution of the number of lines in ARM board les and ARM DTS les 0
100000
200000
300000
400000
500000
600000
v2.6.14
v2.6.19
v2.6.24
v2.6.29
v2.6.34
v2.6.39
v3.4
v3.9
v3.14
v3.19
v4.4
v4.9
v4.14
v4.19
v5.3
arm32 board
arm32 DTS
arm64 DTS
33 / 35

57. Thanks & Acknowledgements
IRenesas Electronics Corporation, for contracting me for
upstream Linux kernel work,
IEmbedded Recipes, for organizing this conference and
giving me the opportunity to present here,
ITheLinux Kernel Community, for having so much fun
working together towards a common goal.
34 / 35

58. Questions?
?
35 / 35