“Federated ML Architecture for Computer Vision in the IoT Edge,” a Presentation from Cisco

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/09/federated-ml-architecture-for-computer-vision-in-the-iot-edge-a-presentation-from-cisco/

Akram Sheriff, Senior Manager for Software Engineering at Cisco, presents the “Federated ML Architecture for Computer Vision in the IoT Edge” tutorial at the May 2024 Embedded Vision Summit.

In this talk, Sheriff begins by introducing federated learning (FL) for computer vision in IoT edge applications. Federated learning is an approach to machine learning that enables collaborative training of deployed models while maintaining decentralized data. He surveys a variety of existing FL architectures and highlights the challenges associated with them, such as statistical dataset issues and system complexities.

Sheriff then describes a novel FL approach that addresses and solves these challenges for computer vision and IoT edge applications. He shares results comparing this novel approach with existing approaches and highlighting its advantages and limitations. He also shows examples of real-world applications where federated learning is used for data privacy reasons, such as in healthcare. You’ll gain insights into leveraging FL for efficient and privacy-preserving model training in IoT-based computer vision systems.

1. Federated ML Architecture
for Computer Vision in the
IoT Edge
AkramSheriff
Senior Engineering Manager AI_ML
Engineering
Cisco Systems

2. •Introduction to federated learning in computervision
•Federated learning architectural patterns for deployment
•Existingfederatedlearning architectural challenges in computer vision
•Proposed federated learning with hybrid models for computervision use
cases
•Advantages ofthe proposed approach and merits of the architecture
•Real world example offederated learning in healthcare computervision
use case.
•Summary and key takeaways
Agenda
2© 2024 CISCO SYSTEMS

3. •Federated learning involves multiple nodes collaboratively training a model in a distributed manner.
•Federated learning normally involves a decentralization of the data by the nodes.
Introduction to Federated Learning in Computer Vision
3© 2024 CISCO SYSTEMS
IllustrationofFLin Computer Vision use case

4. a)Centralized/global federatedlearning
b)Cloud-based distributed federated learning
c)Decentralized federated learning
d)Multi-task with de-centralizedparameter exchangingfederated learning
Federated Learning Architectural Patterns for
Deployment
4© 2024 CISCO SYSTEMS

5. •Unbalanced local datasets:
•Statistical differences in datasets:
•Larger number of worker nodes:
•Heterogeneous Network connectivity:
•Heterogeneous Computer power:
•Data Privacy Concerns
Existing Federated Learning Challenges in Computer
Vision (CV)
5© 2024 CISCO SYSTEMS

6. •For computer vision/CV tasks such as object detection the size of model would be large.
•Data Aggregation, Data sovereignty and Data provenance issues.
•Spatial Data Heterogeneity across the Training Nodes.
More Challenges
6© 2024 CISCO SYSTEMS

7. •FedCV framework is FL topology, architecturevariantsagnostic.
•Ease ofuse FedCVAPI’s
•FedCVis a distributed trainingtoolkitfor analysis, benchmarking,library and platform for
executingCV applications.
•FedCVhelps in bridging gaps between SOTA algorithms and facilitating the development
ofdifferentvariantofFLtechniques.
•FedCVreducesengineering development effortwith multiple embeddedfeatures.
Proposed Federated learning with HierarchicalFLforComputer
Vision(CV) with FedCVframework
7© 2024 CISCO SYSTEMS

8. ProposedHierarchicalFLlearning layer has thefollowing advantages
•By doingthe learning in these smaller Micro-batches based training.
•Nodes then perform small batches oftraining ontheir local data.
•Periodically, each training node submits ML model parameter/weight updates to the central node.
•Holistic view during FL based model weights update andconvergence.
•This process caneither take place indefinitely or be repeated until the FL model converges with
respect to some evaluation metric(e.g., mean average error,accuracy).
ProposedHierarchicalFL Technique
8© 2024 CISCO SYSTEMS

9. Proposed HierarchicalFLTopology
9© 2024 CISCO SYSTEMS

10. •Multi node and Multi layeredarchitecture withFL technique.
•Failure ofoperation ofFL architecture is minimal
•CV application context and data specificsignificance given to the creation ofFL weights.
•Tree basedHierarchical FL improves theconvergence performance.
•The location of aggregator nodes need not be pre-determined in an H-FL architecture which
givesflexibility
•Network Topology specificrouting ofincominginferencing APIrequests.
•No fixed location ofaggregatorand Non-aggregator nodes.
•Aggregator nodes may be dynamically placed within the network to improve model accuracy and
execution performance.
Advantages
10© 2024 CISCO SYSTEMS

11. Performance Improvements Results —FedCV based
Training and Evaluation
11© 2024 CISCO SYSTEMS

12. Problem:
Medical data andHealthcare verticalfaced insurmountable hurdles with patient privacy concerns,data silos,and ethical issues.
Solution:
•Federated learning empowers individual devices and institutions to collaboratively train AI models.
•Federatedlearningoffersnetwork of hospitals,each holding unique clinical datasets.
•Patient privacy, Datasovereignty, Data lineageensured withFL
AdvantagesofFederatedLearningin Healthcare:
•FLcould be usedtoprovideTargetedPrecision medicine for a Patienttocurefrom Fatal diseases.
•Patient'sprivacyensured but at the same time real time data collected and monitored locally in aFLarchitecture.
•Country, RegionspecificMedicaldata Compliancecould be achievedwithFLarchitecture.
•FLscalable across a Global chain ofHospitals, Medical researchInstitutionswithdata loss andensuringdata Privacy.
•Democratization ofVaccine and Medical IPto enable low cost medicine in aspecificregion/Country.
Federated Learning in Healthcare —Real world usecase
12© 2024 CISCO SYSTEMS

13. Federated Learning in Healthcare —ReferenceArchitecture
13© 2024 CISCO SYSTEMS

14. •Federated learning (FL) is a decentralized approach to training machine learning models.
•Federated learning gives advantages of privacy protection, data security, and access to
heterogeneous data.
•Federated learning architectural paradigm complies with data sovereignty norms.
•Federated learning with good architectural patterns can be used for CV use cases.
•Selection of the right FL software framework (FedCV), API’s, hierarchical architectural
design pattern is important for CV use case.
•Ongoing research and industry work in the intersection of FedCVbased FL techniques
and LLM’s to build different Multi-modal LLM applications.
Summary and Key Takeaways
14© 2024 CISCO SYSTEMS

15. References
15
A Field Guide to Federated Optimization
https://arxiv.org/abs/2107.06917
Optimization in Federated Learning
https://ceur-ws.org/Vol-2473/paper13.pdf
Reddi, S. et al. Adaptive Federated Optimization.
Arxiv(2020)
https://openreview.net/forum?id=LkFG3lB13U5
https://www.v7labs.com/blog/federated-
learning-guide
https://arxiv.org/pdf/2308.13558.pdf
https://github.com/OliverStoll/Anomaly-
Detection-IIoT
https://github.com/izakariyya/testbed-fl-iot
https://github.com/FedML-AI/FedIoT
https://github.com/topics/federated-learning
https://github.com/qub-blesson/FedAdapt
https://github.com/qub-blesson
© 2024 CISCO SYSTEMS

16. 16© 2024 CISCO SYSTEMS
Thanks (Q&A)