Intro to Neo4j Ops Manager (NOM)

Intro to Neo4j Ops Manager (NOM)

1. Intro to Neo4j Ops Manager (NOM) Graph Connect, June 2022 Richard Macaskill Costa Alexoglou

2. Richard Macaskill Introducing myself Product Manager Neo4j Core Database Team @datamacas

3. The current challenge Graph database adoption is growing >20% annually Hybrid deployments in clouds and on premises Neo4j continues to grow in features and functions

4. The current challenge Data size is exploding Database size has grown from GB to TB and even PB Neo4j is being scaled out as clusters Pressure on IT Leaders and DBAs to Deliver and maintain applications on time and on budget Pressure to increase productivity Technical resources are expensive in a tight labour market

5. Previous/Existing options 3rd Party Monitoring tools Require setup and integration; separate licenses and support contracts Resulting charts and reports are not within Neo4j context Command line interface for administration and operations Requires familiarity with graph concepts, deployment topology, and Cypher Learning curve can be steep for DBAs new to Neo4j No single place to monitor, administer, and operate your Neo4j estate

6. The Perfect World At the push of a button , view the statuses of all Neo4j databases, clusters, and instances. View the health metrics of each Neo4j database and also of the resources where it is deployed Create databases, configure and administer them at scale without first having to master the graph language. Put important but mundane tasks such as scheduled jobs, backup / restore, upgrades on autopilot Evaluate and monitor query performance. Use recommendations to fine tune performance

7. Administration Operations Monitoring Intelligent Monitoring Knowledge Engine The Solution Pre-integrated, turnkey Monitoring for all Neo4j installations in your estate Status at a glance Metrics dashboard Neo4j at scale, from 1 place Cloud or on-prem Single instance or clusters Monitoring Administration Security management Operations, automated * Built-in Intelligence * Descriptive Analytics Rules-based statuses Prescriptive recommendations * forward looking

8. ‹#› Increased Productivity Faster ramp-up of new DBAs / SREs / Dev Ops Easier to create / configure databases* Simpler maintenance of instances and clusters Less or same overhead What you gain: large scale Neo4j with less headache Improved Reliability Faster issue detection in database or machines Monitor health metrics out of the box and within Neo4j context Optimized Performance* Evaluation and recommendations for faster Neo4j queries Recommendations for more efficient schema * Forward looking

9. Architecture Principles Loose coupling with APIs Thin Client (Web Browser) Feature Frameworks Metadata driven Knowledge Engine Intelligent Monitoring

10. Hands on/Demo

11. 🙋🏻‍♂️ Hi! I’m Costa Alexoglou Senior Software Engineer at Neo4j

12. Agenda Demo Setup What is an agent Monitoring your Neo4j Infra Speed up the Operations Registering an agent

13. Demo Setup

14. What is an agent Small Golang executable Monitors OS and Neo4j related metrics Solves the firewall problem

15. Monitoring your Neo4j Infra Demo time 🎉

16. Demo for Login Home Page Status Panel Dashboard Alerts

17. How the demo is set up NOM server with single instance Neo4j DBMS persistence Already running 1 x 3 core cluster already registered and managed by agent (agent/instances online) 1 x single instance already registered and managed by agent (agent/instance down) Some data loading scripts which can be run on the managed DBMS(s) Additional instance ready to join cluster as read-only during the demo

18. What the demo/workshop will cover NOM in action: a walkthrough of the GUI Home Page Status Panel Dashboards Alerts for DBMS Security Manager Upgrades NOM setup/configuration Adding an instance to NOM

19. NOM in action - Home Page Sign into NOM (let’s not have the warning about common passwords!) Show the home page - system topology NOM, DBMSs and Instances Show single instance agent is offline in topo Show alert message that agent is down on single instance Show DBMS selector in header (and explain the matching names in the Home Topology) Start the agent (only) on the single instance Show the changes in topology Show the changes in alerts Start the single instance DBMS Show the changes in topology Show the changes in alerts Choose a DBMS to work with and move to Status Panel

20. NOM in action - Status Panel Show the system topology view Explain the focus is “next level down” from home page Create a new database - show it in the topology Stop a database - show it in the topology Drop the new database - show it in the topology Switch to tabular view Same output as cypher “show databases” Demo sort and search on this table [No need to show database view] Move to Dashboard

21. NOM in action - Dashboards (Host) Explain title bar (period selector, tabs) Table for hardware resources Quick run through the host metrics Explain the “polling interval” and “must poll” logic 30secs by default with 5 min “must poll” Mention that housekeeping is coming in a future release With cluster selected, show the member selection with legend ([en|dis]able) Move to dashboard -> instance tab

22. NOM in action - Dashboards (Instance) Show table of disk resources (Explain that directories share same mountpoint?) Quick run through the instance metrics Move to Dashboard -> Database tab

23. NOM in action - Dashboards (Database) Show database selector (Explain that database dashboard is for a selected database) Quick run through the database metrics Move to DBMS alerts

24. NOM in action - DBMS Alerts Explain that this is a subset of the home page alerts for the selected instance Stop one of the cores on the cluster with the cluster selected (in time) show that the instance down alert appears Move to Security Manager (Here I can display the Warning of the ghost instance and pass to the operations chapter, and start on how to solve this alert by creating and starting a new agent inside the VM of the ghost instance)

25. Speed up Operations Demo time ⚙️

26. Demo for Creating and setting an agent -> Then display the alert disappearing and the VM appearing in the topology Security Manager Upgrade About?!

27. NOM setup - register an agent Register an agent Show the instructions to install the agent properly Start the agent already installed in the server Display the alert removal in the Alerts Page Display the new DBMS instance in the topology

28. NOM in action - Security Manager (Users) Show how to create a user Also show the cypher code Show how to edit a user Show how to reset a user’s password Show how to remove a user Mention that some user capabilities will be added soon (homeDB, rename) Move to Security Manager -> Roles

29. NOM in action - Security Manager (Roles) Show how to create a role Show how to edit a role Membership Rename workaround Show how to clone a role Show how to remove a role Move to Security Manager -> Privileges

30. NOM in action - Security Manager (Privileges) Show how to grant a new privilege Show how to deny a privilege Show how to revoke a privilege Use filters to find privileges quickly Move to Upgrades

31. NOM in action - Upgrades Show the available information in upgrades Explain the functionality which will be added for Neo4j 5 migration And other upgrades Clarify that fully automatic upgrades are not (yet) on the roadmap

32. NOM setup - add an instance To demonstrate NOM setup, we will walk through adding a read replica to the cluster Start the prepared RR instance and let it join the cluster and sync raft groups Show the alert that there is an unmanaged instance in the cluster If RR instance is on same host, add the instance to the agent If RR instance is on a new host, deploy and configure the agent Show that the instance is now shown and the alert is removed Show that the Home page topology, status panel and dashboard are all updated

33. Useful links https://prometheus.io/docs/prometheus/latest/configuration/configuration/ https://neo4j.com/docs/operations-manual/current/monitoring/metrics/reference/ https://neo4j.com/docs/operations-manual/current/docker/clustering/

34. Next Steps

35. Neo4j Operations Manager 1.0 and next priorities Support for multiple Neo4j DBs Support for self-managed on prem and cloud deployments Topology and statuses at a glance Metrics dashboard Up / Down alerting Security management (RBAC) Docker / Kubernetes deployments Upgrade and Migrate License management Logs, Query , and Performance Forward looking v 1.0 GA Configuration management Update every 8 weeks: fixes, functions, features

36. Get it - timeline, prerequisites and eligibility Available end June 2022 Manages Neo4j Enterprise Edition instances running 4.4.0 and later No support for Community Edition No support for versions earlier than 4.4.x Works currently with Windows and Linux tarball/zip installations Package managers should be available at launch Docker/Kubernetes should be available shortly after launch Available FOR FREE for customers with a valid Neo4j EE DBMS license

37. Q&A Session FAQ responses Does it work with Aura, Cloud? Support for Aura (managing Aura instances and running NOM as a Service) will be added later NOM will be able to be run on AWS, GCP, Azure Does it work with Docker/K8s? Not currently, but support is expected early Q3 2022. Does it work with CE? No support for CE is currently planned Does it work with Desktop? No support for desktop is currently planned How much does it cost? It is free to use for a customer with a valid EE Neo4j DBMS license Can I try this out without an EE license? A 30 day evaluation is possible Can I manage a DBMS without an agent? No support for agentless NOM is currently planned When is it available and how do I get it? Released by the end of June, will be available to download from neo4j.com