CloudMonitor - Architecture Audit Review February 2025.pdf

CloudMonitor FinOps is now a Microsoft Certified solution in the Azure Marketplace. This little badge means that we passed a 3rd-party Technical Audit as well as met various sales KPIs and milestones over the last 12 months.

We used our existing Architecture docs for CISOs and Cloud Architects to craft an Audit Response - I've shared it below to help others obtain their cert.

Interestingly, 90% of our customers are in the USA, with very few in Australia. This is odd as the first thing I hear in every meetup and conference, from partners, customers and Microsoft, is that they want to optimise their cloud spend! But very few Australian companies are using the FinOps Framework to lower Azure costs.

1. Architecture Audit Review – February 2025
Azure FinOps Managed Application to control cloud spend
Commercial in Confidence – Do not distribute beyond the intended audience.
Version 1.0

2. 1. Architecture

3. Architecture
•No IaaS
•PaaS only
•Serverless where possible
•All telemetry captured to Monitor
•PowerBI SaaS Frontend
•Walkthrough Video

4. Key Actors, Users & Service Principals
CloudMonitor installs via the Azure Marketplace using the Managed Azure Applications deployment model (A similar model that Databricks uses).
Component Purpose Notes
CloudMonitor Service PrincipalCommunicates with Azure APIs. Read-only on Service Level data but has ability
to read Billing Data and create Cost Export Schedules.
Created/Managed by Customer
Customer The purchaser of the CloudMonitor Product. CloudMonitor is installed in the Customer’s Azure
Tenant.
CloudMonitor Support The CloudMonitor Helpdesk. CloudMonitor is a “Managed Application” and our
support team need to upgrade the application to new
versions and support the product.
Vendor The owners of the CloudMonitor Product CloudMonitor Services PTY LTD, headquartered in
Sydney, Australia.
CloudMonitor The Vendor’s Intellectual Property (IP). Usually referring to the Azure Managed
Application.
Made up of various components but refers to the whole
solution.
Microsoft Azure MarketplaceHosts CloudMonitor and handles the Billing arrangements and forex
conversions. Microsoft designed and manage the deployments of the Azure
Managed Application from the Marketplace to the Customer’s Azure Tenancy.
The Customer purchases the product from Microsoft
Azure Marketplace, not directly from the Vendor.
Microsoft has a supplier agreement with the Vendor and
the Customer and is the glue between the two.

5. 2. Solution Demonstration

6. Solution Demonstration Videos
Click on the links below to watch the videos
Architecture Walkthrough Video:
https://www.loom.com/share/28601ce7ed32418597c54c06addf4177?sid=81e55b1a-918e-4c3d-97c5-
4722daa5dcbb
Demo Video of CloudMonitor: CloudMonitor - Extensible FinOps Platform for Azure

7. CloudMonitor FinOps Solution on Azure
CloudMonitor is a Managed Azure Application that runs as a single-tenanted architecture inside of the customer’s tenancy
CloudMonitor
FinOps
CloudMonitor
Security & Compliance
CloudMonitor
Microsoft 365 Usage/Licensing
CloudMonitor
Sustainability
CloudMonitor
Admin App
CloudMonitor
Teams Bot
(Alerting)
Analytics Engine
FOCUS Billing Data

8. The CloudMonitor Solution installs inide of your Azure cloud Tenancy. You control the policies.
YOUR
Azure Active Directory
Power BI
CloudMonitor Admin App
Cost Groups and Audit Logging
CloudMonitor Analytics Engine
Azure Managed
Resource Group
Azure APIs
Read-Only
Service Principal
CloudMonitor Power BI Reporting
Cost Visibility and Dashboards
CloudMonitor Teams Bot
Dev Workflow
Recommendations & Alerts
Your Azure Subscription
Your Cloud/Organization
High Level Solution
Click here to read FAQs on how CloudMonitor works

9. Primary Solution Components
The CloudMonitor managed solution is made up for 4 key components:
Component Purpose Notes
CloudMonitor Analytics EngineOur FinOps IP that converts raw cost data and Azure telemetry into useful
analytics, recommendations and actionable insights.
TheEngine is installed from the Azure Marketplace.
CloudMonitor Power BI/Fabric
Reports
The Business Intelligence (BI) tool for displaying the FinOps Analytics,
dashboards, KPIs and trends. Power BI is read-only.
The Power BI App is installed from the AppSource
Marketplace.
CloudMonitor Admin App The application to create Cost Groups, take actions on recommendations, and
set up Budgets.
All authentication is done via Azure Entra / AD
CloudMonitor Teams Bot Provides near real-time cost anomalies and alerts as part of your workflowAll authentication is done via Azure Entra / AD

10. 3. Azure Advisor Score

11. CloudMonitor use Azure Advisor
Screenshot taken from a production CloudMonitor Install

12. 4. Well-Architected Framework Review

13. CloudMonitor WAF Review
CloudMonitor - Azure Well-Architected Review - Feb 26, 2025 - 9:01:34 PM - Assessments | Microsoft Learn
CloudMonitor - Azure Well-Architected Review - Feb 26, 2025 - 9:01:34 PM - Assessments | Microsoft Learn

14. CloudMonitor WAF (In our Product)
One of the product features in CloudMonitor is to help customers understand their OWN Secure Score, and WAF posture. Here are some example
screenshots from an anonymous Customer who is using CloudMonitor:

15. CloudMonitor WAF (In our Product)
Product screenshot of a Customer’s Secure Score

16. CloudMonitor WAF (In our Product)
Product screenshot of a Customer’s Secure Score

17. 4. Cloud Security Posture Management

18. Microsoft Defender for Cloud
CloudMonitor uses Microsoft Defender for Cloud as Cloud Security Posture Management (CSPM) platform

19. Defender - Security Posture

20. Defender - Recommendations

21. 5. Technology Specific Questions
2A - Azure Data Services

22. 2A.1 Azure services interoperated or utilized
The CloudMonitor solution utilizes and interoperates with the following Azure PaaS services:
•SQL Database
•Storage Account
•Data Lake (ADLS2)
•Synapse Workspace
•Power BI

23. 2A.2 Data processing handled within Azure
CloudMonitor performs all data processing or transformation operations on Azure hosted services.
PenTests provided in Security Section
See the Data Flow within Customer Tenancy: CloudMonitor IP (Managed Resource Group)
CloudMonitor Frontend
Non-Managed
Resource Group
Common Services
Key VaultAdmin
Storage
Application
Insights
Power BI
Reports
SQL Database
Customer
Azure
Active Directory
Azure
Application IP
Bot Service
Teams
Bot
Log Analytics
Workspace
Outlook
Alert
Queue
Azure
DevOps (IaC)
Admin App
Synapse
Export Landing
Zone
Azure APIs
FOCUS Billing Data
Data Lake
Jira/
ServiceNow/
Other ITSM
Master Data
Notifications

24. 2A.3 Customer data is gathered & stored in Azure
Any end-customer data that is gathered is stored in Azure in a Data Lake in a Managed Resource Group.
CloudMonitor provides customers with the capability to analyze data from Power BI and from a custom static
web app called AdminApp hosted in customer’s tenant.
CloudMonitor Engine and Power BI reports are installed from the Azure Marketplace:
•CloudMonitor Cost Analytics - FinOps Platform (FOCUS V1.0)
•CloudMonitor Enterprise FinOps (FOCUS) - Azure Cost Management & Optimization
Overview of the type of customer data stored: CloudMonitor - Extensible FinOps Platform for Azure

25. 2A.4 Data access patterns for integration
Function showing connection to the Pricesheet API retrieval, logging and writing to the SQL Database

26. 2A.5 Data use for AI/ML model generation
CloudMonitor does not generate AI/ML models from customer data.

27. 2A.6 Data encrypted in transit and at rest
All data is encrypted at rest and in transit .
TLS 1.2 and all native PaaS encryption settings are set to the latest in IaC:

28. 5. Technology Specific Questions
2B - Azure AI or Machine Learning Services

29. 2B.1 Azure services interoperated or utilized
The CloudMonitor solution utilizes and interoperates with the following Azure services:
•AI Bot Service

30. 2B.2 Data section requirements must be followed
CloudMonitor has met all the requirements of Azure Data Services section

31. 2B.3,4,5,6 Model operations must occur on Azure
No ML is used in our solution

32. 5. Technology Specific Questions
2C - Azure Compute Services

33. 2C.1 Azure services interoperated or utilized
The CloudMonitor solution utilizes and interoperates with the following Azure services:
•App Services
•Functions (Durable and Non-Durable)

34. 2C.2,3,4 Azure services interoperated or utilized
No IaaS (VMs) are used in CloudMonitor.

35. 5. Technology Specific Questions
2D - Azure Container Services

36. 2D.1 Azure services interoperated or utilized
The CloudMonitor solution does not include Azure KubernetesService, Azure Container Apps, Azure Container
Instances.

37. 5. Technology Specific Questions
2E – Azure Integration Services

38. 2E.1 Azure services interoperated or utilized
The CloudMonitor solution utilizes and interoperates with the following Azure services:
•Azure Queue Storage

39. 2E.2 Integration patterns
C# code showing use of Storage Queue SDK

40. 2E.3 Data processing handled within Azure
See section 2A.2 for data flow and integration

41. 2E.4 Customer data is gathered and stored in Azure
Refer to A2.3 Customer data is gathered and stored in Azure for Power BI Reports

42. 5. Technology Specific Questions
2F – Azure Control Plane Services

43. 2F.1 Provisioning and/or managing Azure services
The Azure Application is deployed to a Managed Resource Group which the customer cannot Edit due to RBAC
Deny permissions enforced during marketplace deployment. The Data-Driven Tenancy (Creators of CloudMonitor)
has a Service Principal that is added with Owner as per our Marketplace Listing. We can therefore manage and
upgrade our product in the customer’s Tenancy.
See Architecture Diagram.

44. 2F.2 Resource organization (1)
We use ARM deployment and deploy as a Managed Resource Group into the Customer’s Subscription.

45. 2F.2 Resource organization (2)
Our product allows the user to configure a Read-Only Service Principal for the Subscriptions that they want to
monitor. They can add/delete at will.

46. 2F.3 Identity
We follow the principal of Least Privilege as per our ISO 27001 Certification.
Active Directory and RBAC is used in all cases and MSI between components, if supported. All of our deployments
are down view Service Principal and IAC.
We attest that all environments are managed in the manner specified.

47. 2F.4 Policy (1)
The deployment mechanism in Azure Managed Apps is limited by RBAC and it is impossible to alter Policies
outside of the scope of the Managed Resource Group. We do not enforce or edit any policies in CloudMonitor.

48. 2F.4 Policy (2)
The deployment mechanism in Azure Managed Apps is limited by RBAC and it is impossible to alter Policies
outside of the scope of the Managed Resource Group. We do not enforce or edit any policies in CloudMonitor.

49. 2F.4 Policy (3)
The deployment mechanism in Azure Managed Apps is limited by RBAC and it is impossible to alter Policies
outside of the scope of the Managed Resource Group. We do not enforce or edit any policies in CloudMonitor.

50. 2F.5 Deployment operations (1)
Deployments are done via Azure DevOps, IaC and ARM. All logs for CloudMonitor installations are stored in the
ARM template against the Managed Resource Group in the customer’s tenancy, including success and failures.

51. 2F.5 Deployment operations (1)
Detailed Deployment log:

52. 2F.5 Deployment operations (2)
The ARM Deployment logs are native Azure ARM functionality, and all logs are stored (See previous screenshot of
aged deployment)

53. 2F.5 Deployment operations (3)
Customers have READ RBAC on their own Managed Resource Group and can hence access the same logs during the
retention period:
https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2Fd348ceb
2-9ee2-493f-8396-36ddea08c344%2FresourceGroups%2Fmrg-CloudMonitor-DEV006-CSP-
CMDevTenant%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2FmainTemplate-20241024-010227-c73f

54. Security

55. CloudMonitor was pen-tested and certified by an independent, 3rd party Security Auditor
Security Audit
•Managed Resource Group – Security Attestation
•PenTest – Executive Summary Report (Available on Request)
•PenTest – Detailed Summary Report (Available on Request)

56. Permissions Required During Installation
The following permissions are required for the installing user in your team during the installation (These are NOT required by CloudMonitor)
Install Action Who Can Do It Notes
Install CloudMonitor Analytics Engine from Marketplace into Azure Resource
Group
Subscription Owner A new Resource Group will
be created
Creation of CloudMonitor Synapse Azure B2B User in MS Entra TenantEntra Admin or Cloud Application
Administrator
The B2B user can ONLY see
the Managed Application.
Configuring of CloudMonitor Engine and Synapse Subscription Owner
Choosing the Management Group or Subscriptions for CloudMonitor to
monitor
Subscription Owner or Management
Group Owner
Only READER role is assigned
to CloudMonitor
Assigning Billing Account READER role (MCA only) Global Azure Admin
Assigning Billing Account READER role (EA only) Department Administrators/Account
Owners

57. Security FAQ
Raise any issues on our Support Desk here.
Question Answer
Is all user authentication done with Active Directory?Yes, to log in to CloudMonitor you need to adhere to the Azure Active Directory / Entra security policies set by your organization.
The CloudMonitor Admin chooses who can access CloudMonitor and grants access to the Power BI reports via the standard Power BI sharing mechanism.
Users log in to Power BI using standard Azure AD Authentication and Authorization is handled by your Azure Active Directory. We recommend enabling MFA
and following standard best practices for configuring Azure AD.
Is my data in CloudMonitor encrypted? Yes, we use the standard Azure security controls to encrypt your data at rest (in Storage Accounts and SQL Databases) as well as in transit over HTTPs.
Read more about how your data isencrypted at rest here. Read more abouthow Azure encrypts data in transit here.
Why do you need an AD B2B User on our MS Entra
Tenancy?
This is due to a documented bug (Ticket: 2201270060004308) in Microsoft Synapse when deployed via a Managed Application in the Marketplace. The
Synapse Workspace incorrectly authenticates with the local MS Entra Tenancy and not the CloudMonitor Vendor Tenancy.
The CloudMonitor Managed Service Principal is not on the local Entra instance and hence authentication fails. As a temporary workaround, we require a
local account for managing the Synapse instance in the Managed Resource Group. Once Microsoft fixes the bug in Synapse this will no longer be required
and we can use the standard Vendor Managed Service Principal to do this and the B2B user can be removed.
Note: The B2B user has very limited rights on the CloudMonitor Managed Resource Group only. All logins and actions will be audited by the Customers AD
Policies.
Why permissions does the AD B2B User need? Synapse Admin Role on the CloudMonitor Synapse Workspace inside of the CloudMonitor Application
Storage Blob Data Contributor on the Storage Accountinside of the CloudMonitor Application
Why do you need my IP address during installation?This is due to the Microsoft bug above. It allows us to grant your user the correct permissions during installation by whitelisting your IP temporarily. This
whitelisted IP is removed during the same install process once it is configured correctly.
Do you have an Information Security Policy? Yes, this can be provided on request. It is reviewed annually and included in staff training.
What happens in the event of a security breach in
CloudMonitor?
We have a Data Breach / Incident Response policy that guides our action. Customers will be notified as per our policy.

58. Data Privacy

59. Data Security Principles
Your cost and resource data does not leave your organization. CloudMonitor has read-only access at the service-plane level. You choose what CloudMonitor can see.
Keeping client data secure is our top priority.
We value your trust and have implemented an extensive system of security controls and practices to ensure that your information and data are secure.
Your data never leaves your system.
Our CloudMonitor IP leverages the Azure Managed Application deployment model to ensure that all data and analytics remain inside of your tenancy.
CloudMonitor cannot read the datainyour services – only cost and metadataaboutservices
For example, it cannot read secrets in key vaults, data and schema objects in SQL Databases or data on Storage Accounts (or any other Azure Service).
CloudMonitor cannotupdateanything in your Azure estate
It is physically impossible for CloudMonitor to make any updates to any Azure Resources via Azure native RBAC controls (Reader only)
Strict Access Control is a Must
We employ significant standard controls to ensure your data remains secure. CloudMonitor actively employs a policy of least provisioning, where employees are only granted the
minimum system access to perform their assigned job functions. Active Directory and cloud-native authentication and authorization are used by default to allow full auditing.
Our Cloud Partners are ISO27001 and SOC Compliant
CloudMonitor stores its data in Microsoft Azure data centers. All of the data centers have achievedISO/IEC 27001andSOC 1certification, PCI DSS Level 1 compliance.
Our Information Security Policy handles risk scenarios (provided on request)
CloudMonitor believes it is essential to investigate all potential vulnerabilities. We have clearly defined procedures to fully investigate all reported issues.
All data is encrypted – At rest and in transit
All information that we receive and transmit is fully encrypted. We are committed to using the most advanced encryption techniques to ensure that you are as protected as possible.
You choose in which region your data is stored.

60. Key Datasets used in CloudMonitor
CloudMonitor is unique in that it merges Cost information with service-level information to provide context to recommendations, anomalies and analytcs
Dataset Notes Extraction
Actual Costs The actual costs incurred financially at the time of purchase. Scheduled Cost Export to Storage Account
Amortized Costs Contains amortization data for purchases such as Reservations. Scheduled Cost Export to Storage Account
Subscription Metadata All metadata related to the Subscription at the service level (tags, activity log, resource
group, location, name, created by person etc).
Ingested via Azure Functions / Service Principal
ResourceGroup Metadata All metadata related to the Subscription at the service level (tags, activity log, resource
group, location, name, created by person etc).
Ingested via Azure Functions / Service Principal
Resource Metadata All metadata related to the Resource at the service level (tags, activity log, resource
group, location, name, created by person etc). CloudMonitor cannot read any data at
the data plane level – e.g. inside of databases, storage account contents or data.
Ingested via Azure Functions / Service Principal
Billing Account / ProfileInformation about the type of Billing Account used by the Azure Tenancy.Ingested via Azure Functions / Service Principal
Azure Advisor RecommendationsThese are merged with custom recommendations and put through the CloudMonitor
notification engine and sent to the correct Owners at the right time.
Ingested via Azure Functions / Service Principal
Reservation Transactions/DetailsAll RI details used to provide RI analytics and recommendations Ingested via Azure Functions / Service Principal
Secure Score The Security Dashboards display vulnerabilities and improve the security posture.Ingested via Azure Functions / Service Principal
Security Recommendations The Security Dashboards display vulnerabilities and improve the security posture.Ingested via Azure Functions / Service Principal
Last User Login Details * Optional: CloudMonitor can notify/report on expiring service principals, if configured.Ingested via Azure Functions / Service Principal
Expiring Service Principals *Optional: CloudMonitor can notify/report on stale users, if configured.Ingested via Azure Functions / Service Principal
Microsoft 365 Datasets Optional – M365 reports can be disabled, e.g. User Details, Ingested via Azure Functions / Service Principal
* Optional with configuration. These features can be turned off.

61. PII Data – Email Address
CloudMonitor displays the Created By User and Owner for all Cost Groups, Subscriptions, Resource Groups, and Resources
Dataset Notes
Last User Login Details * Optional: CloudMonitor can notify/report on expiring service principals, if configured.
* Optional with configuration. These features can be turned off.
A foundation principle of FinOps is delegation of responsibility down to the business units and accountability of costs by these teams.
In order to drive accountability and governance, CloudMonitor records the Created By Email Address and Owner for various entities. These are then used by the Teams Bot and for
emailing of Budget, Recommendation and Cost Anomaly alerts.
The following datasets expose data that could be considered PII and the cost/benefit of these reports should be considered. In the screenshot below, the “Last Login” functionality
has been disabled.

62. How We Access Your Data
A Managed Application is many times more secure than sending your data outside of your organization without any controls to secure it outside.
Technically, our Support Team could view the datasets specified in the “Key Datasets” slide. CloudMonitor is a Managed Application and a complex Big Data tool. Our Support Team
can debug and fix any issues in the data processing pipeline without affecting you as the end user. We also upgrade our product frequently as we add more enhancements.
Rest assured we have no intention of viewing your data beyond the requirements for supporting the product. We would damage our brand and our reputation if we violated this.
Our larger Enterprise and Government customers usually request that we sign an NDA or other relevant contract, depending on location and security requirements to legally
mitigate this risk. We are happy to review and agree to any reasonable request.
Our support staff use a dedicated CloudMonitor Azure Virtual Desktop through a VPN. This AVD has a dedicated IP that can be whitelisted, if required. All staff are bound by our
Cybersecurity and Information Security Policies which are reviewed frequently.
Please email any security or data privacy concerns or questions to [email protected] and we will address them.

63. Configure the Security Access Model to suit your business with Cost Group Roles
CFO/CIO/CEOs
Overall Executive Summary and KPIs
Project/Program Managers
KPIs/Summary per Project (Cost Group)
Engineers / Consultants
Creating Resources at Project Level (Cost Group)
Cost Group Viewer
Cost Group Editor
Cost Group Admin
Cost Group Viewer
All Cost Groups
Specific Cost Groups
Specific Cost Groups
Power BI Reporting
Application
Teams/Slack Bot
Create Cost Groups
Assign Users
View Cost Reports
View Cost Reports
Top Insights
Act on Recommendations
Monitor Cost Anomalies
Create Audit Trails
Receive Alerts
Example Access Configuration
You control who can see what in each Cost Group with RLS

64. Data Privacy FAQ
Raise any issues on our Support Desk here.
Question Answer
Where is my data hosted? Your raw data is stored in the Managed Resource Group in the Subscription that you choose on your Azure cloud tenancy when
you install CloudMonitor (ie. it is inside ofyourAzure tenancy). The region of this Resource Group (which is nominated by you at
install time) determines in which Azure Data Center your data will physically be stored. We recommend that you choose the usual
region closest to where your business users are to achieve the best performance.
Does my cost data ever leave my cloud
tenancy or organization?
No, we are not a multi-tenanted SaaS product and therefore do not need to extract your data. All of your analytics are created
inside of your cloud where CloudMonitor is installed.
What can you see inside of my Azure tenancy?At a Subscription and Resource Group level, weonlyare able to see the Managed Resource Group containing our CloudMonitor IP.
This is made up of Azure services that process your raw cost and service metadata into recommendations and useful analytics. We
do not have access to any other Resource Groups or internal systems on your cloud estate.
The Service Principal that you provide has READER access to extract costs and resource metadata (it cannot read the data inside
the resources, e.g. on SQL Databases or Storage Account data) – we use these APIs to build up the Resource Graph of your system
and the associated costs.
Can you read my customers’ data? No, a Service Principal with the READER RBAC role cannot read the data inside of Azure services like key vault secrets, storage data
or SQL Database data. It is physically impossible. CloudMonitor only needs access to the Cost and metadata on the Azure service
plane, not the data plane.
Can the CloudMonitor Support team see any
data in any of my resources?
Absolutely not. The Reader RBAC role assigned at the Subscription level can only read Service Plane data, not any data inside of
the resources (e.g. we cannot read data on your databases, keys in your key vaults or data on your storage accounts).

65. Deployment Model

66. Managed Azure Applications
CloudMonitor installs via the Azure Marketplace using the Managed Azure Applications deployment model (A similar model that Databricks uses).

67. Managed Azure Applications – Key Concepts
Managed Applications are supported and upgraded by the Vendor inside of the Customer’s Tenancy. No data leaves the Customer’s domain.
Microsoft documentation on Azure Managed Applications
Component Purpose Notes Managed By
Application Resource
Group
Contains the Managed
Application record
Customer can delete the application
from here at any time
Customer
Managed Resource GroupContains Vendor’s IP – a
mixture of PaaS Services
such as Storage Accounts
and data storage
No changes can be made to our IP by the
Customer. The Vendor updates the
platform to new versions and supports
the data platform. The Vendor can ONLY
see these resources inside of this
Resource Group and nothing outside on
the Subscription.
Vendor

68. Questions?
Contact Us!