Business intelligence unit 3 ppt. It explored working of BI tools

Business intelligence is an elective subject in Dbatu longer university and these are ppts for the same

1. Working with BI Tools Dr. S V Balshetwar

2. Difficulties of Implementing Data Warehouses Implementing a data warehouse is generally a massive effort that must be planned and executed according to established methods. Construction, administration, and quality control are the significant operational issues which arises with data warehousing. Some of the important and challenging consideration while implementing data warehouse are: the design, construction and implementation of the warehouse. The building of an enterprise-wide warehouse in a large organization is a major undertaking.

3. Manual Data Processing can risk the correctness of the data being entered. An intensive enterprise is the administration of a data warehouse, which is proportional to the complexity and size of the warehouse. The complex nature of the administration should be understood by an organization that attempts to administer a data warehouse. There must be a flexibility to accept and integrate analytics to streamline the business intelligence process. To handle the evolutions, acquisition component and the warehouse’s schema should be updated.

4. A significant issue in data warehousing is the quality control of data. The major concerns are: quality and consistency of data. Consistency remain significant issues for the database administrator. One of the major challenge that has given differences in naming, domain definitions, identification numbers is Melding data from heterogeneous and disparate sources. The data warehouse administrator must consider the possible interactions with elements of warehouse, every time when a source database changes. There should be accuracy of data. The efficiency and working of a warehouse is only a good as the data that support its operation.

5. Usage projections should be estimated conservatively prior to construction of the data warehouse and should be revised continually to reflect current requirements. To accommodate addition and attrition of data sources, the warehouse should be designed. This also avoids a major redesign. Sources and source data will be evolve, and the warehouse must accommodate such changes.

6. Another continual challenge is fitting of the available source data into the data model of the warehouse. This is because requirements and capabilities of the warehouse will change over time as there will be a continual rapid change in technology. A far broader skills will be required by administration of data warehouse for traditional database administration. Managing the data warehouse in large organization, design of the management function and selection of the management team for a database warehouse are some of the major tasks.

7. Some best practices for implementing a Data Warehouse: The data warehouse must be built incrementally. User expectations about he completed projects should be managed. It is important to be politically aware. There should be a build in adaptability. Developing a business/supplier relationship is the best practice.

8. Implementing a data warehouse can be a complex and challenging process that involves several difficulties, including:

9. Data Integration:  Data warehouses are designed to integrate data from various sources, which can be a complex process. The data may be stored in different formats, have different levels of granularity, or use different data models. Integrating this data into a cohesive and consistent data warehouse can be challenging. Data Quality:  Data quality is crucial for the success of a data warehouse. Poor data quality can lead to inaccurate or incomplete analyses, which can have significant impacts on business decisions. Ensuring data quality requires careful data cleaning and validation, which can be time-consuming and challenging.

10. Data Volume:  Data warehouses can contain vast amounts of data, which can make it challenging to manage and process. Managing the volume of data requires careful planning, design, and optimization to ensure that the system can handle the required workload. Performance:  Data warehouses must provide fast query response times to support business intelligence and analytics. Achieving high performance can be challenging, as data warehouses require complex data models, indexing strategies, and query optimization techniques.

11. Security:  Data warehouses contain sensitive data, and ensuring data security is crucial. Implementing robust security measures, such as access control, data encryption, and data masking, can be challenging, especially when dealing with large volumes of data. Business Requirements:  Designing and implementing a data warehouse that meets business requirements can be challenging. Business requirements can be complex, and may require specialized data models, analytics, or reporting capabilities. Meeting these requirements requires careful planning, communication, and collaboration between business stakeholders and IT teams.

12. implementing a data warehouse can be a complex and challenging process that requires careful planning, design, and execution. Overcoming these difficulties requires a combination of technical expertise, project management skills, and effective communication and collaboration between IT teams and business stakeholders Cost:  Implementing a data warehouse can be expensive, with significant costs associated with hardware, software, and ongoing maintenance. The cost of data warehouse implementation needs to be weighed against the expected benefits of improved decision-making and increased efficiency. Change management:  Data warehouses are designed to support business decision-making, and therefore, changes in business processes and requirements must be reflected in the data warehouse. Effective change management processes must be in place to ensure that the data warehouse can adapt to evolving business needs.

13. Advantages: Improved Data Quality:  Implementing a data warehouse can help to improve data quality since it involves cleaning and integrating data from various sources. This ensures that the data used for reporting and analysis is accurate, consistent, and complete. Better Decision-Making:  A data warehouse provides a central repository for data that can be used for reporting and analysis. This allows business users to make better-informed decisions based on data-driven insights. Historical Analysis:  A data warehouse stores historical data, which allows users to analyze trends and patterns over time. This can help to identify opportunities and potential issues before they become critical. Scalability:  A data warehouse can be scaled horizontally or vertically to accommodate larger volumes of data or to support more users. Supports Self-Service Analytics:  A data warehouse can be integrated with self-service analytics tools, such as Tableau, Power BI, or QlikView, which allows business users to analyze and visualize the data themselves without relying on IT.

14. Disadvantages: Time and Cost:  Implementing a data warehouse can be time-consuming and expensive since it involves collecting, cleaning, and integrating data from various sources. This can require significant investment in hardware, software, and personnel. Complexity:  Implementing a data warehouse is a complex process that requires expertise in data modeling, ETL processes, and database management. This can make it challenging for organizations to implement and maintain a data warehouse. Data Silos:  A data warehouse can create data silos since it typically requires a separate data infrastructure from the operational systems used by the business. This can make it challenging to ensure that the data in the data warehouse is up-to-date and consistent with the operational systems. Performance:  A data warehouse can impact the performance of the operational systems used by the business since it typically requires a significant amount of processing power and storage. Data Governance:  Implementing a data warehouse requires robust data governance practices to ensure that the data is managed and used in a consistent and compliant manner. This can require significant investment in policies, procedures, and personnel.

15. 10 Key challenges to consider while planning a data warehouse Planning a data warehouse requires careful consideration of various challenges that can impact the success, efficiency, and longevity of the system. Here are the key challenges: Data integration Data quality Scalability Performance Data security Data modeling Historical data handling Regulatory compliance Cost management Change management

16. CHALLENGES FACED IN DATA WAREHOUSE DEPLOYMENT Developing and deploying a hectic task as mentioned earlier and issues are complications are bound to arise out of such large scale tasks. Following are the issues that are faced generally A. Roles and Responsibility Crisis A data warehouse project team might take the time to figure out the roles and responsibilities. B. Data Issues Issues in mapping the data back to its origin can be a complicating task. C. Infrastructure Issues Environmental and Hardware issues are prominent D. Architecture Issues Issues involving software, hardware, and the environment.

17. CHALLENGES FACED IN DATA WAREHOUSE MAINTENANCE Maintenance is a continuous task. Data warehouses are huge in size. An estimated 20% of the time is spent on data extraction, cleansing and loading process. After deployment, the users demand and expectancy increases and it becomes a challenge for the data warehouse team. Following are the issues that are generally faced

18. CHALLENGES FACED IN DATA WAREHOUSE MAINTENANCE Contd … A. Change in Business Process/Policy If there is a change in the business process and policy in the organization, the Data warehouse teams may have to reorganize and the source system may require the data warehouse to change. updates are required in such cases. B. Taking a decision on the data. It becomes a hefty task for the data warehouse team to decide on the data sources and also which data to keep and what not.

19. C. Deciding on the Queries and Access One of the primary task for the project team is to identify the queries that are required for data extraction. Some of the queries are inbuilt and some require users input. The task of deciding the same can be confusing and might create complications in the future. Providing users access to custom queries is also a challenging task and requires security policies in place. D. Uncertainty over Warehouse and Operational Store Users are uncertain in determining if the data should be fetched from the data warehouse or operational store.

20. CHALLENGES FACED IN DATA WAREHOUSE MAINTENANCE Contd … E. Updates need to be applied As mentioned earlier, a change in business policy or a new loophole might require a new update or patch to be applied to the system. F. Architecture Maintainance Maintaining the architecture of the data warehouse is more difficult than establishing a new one. Variables such as vendor support and updates play an important role.

21. G. Security Security of a data warehouse is as important as protecting your own child. The huge amount of data stored in the data warehouse makes it an easy target for unauthorized access. Loopholes and breaches can be imminent and needs to be addressed during the development phases. A change in security policies may require the data warehouse security to be changed or overhauled.