stastistics bcom fintech importants .pdf

jj

1. BCom Fintech SMADA Important Concept Notes

Data Cleaning in Pandas.
Data cleaning is a very important step in real-world data analysis because raw data often
contains errors like missing values, duplicate records, or wrong data formats. These errors
can lead to incorrect results and poor decisions if not fixed. For example, in analyzing sales
data from different stores, missing sales figures or duplicated transactions can cause wrong
conclusions about profits or customer behavior.
Importance of Data Cleaning:
1. Improves Accuracy: Clean data ensures the analysis and models are based on correct
information.
2. Better Decisions: Helps businesses make informed decisions like stock management
and marketing strategies.
3. Prevents Errors: Fixing data errors early avoids problems later in analysis or reporting.
4. Saves Time: Automated cleaning with tools like pandas speeds up the process.

Techniques in pandas:
a) Handling Missing Values:
Missing values can be handled by either filling them with a specific value or removing the
rows with missing data.
1. Use df.isnull() to find missing values.
2. df.fillna(value) fills missing values with a constant or a statistic like mean.
3. df.dropna() removes rows or columns containing missing values.
Example:
In a sales dataset, if some sales amounts are missing, filling missing values with the average
sales helps keep the data complete.

b) Removing Duplicates:
Duplicates can give biased results.
1. Use df.duplicated() to identify duplicates.
2. Use df.drop_duplicates() to remove duplicate rows.

2. Example:
If the same transaction is recorded twice, dropping duplicates avoids counting it twice in total
sales.

c) Correcting Data Formats:
Data from different sources may have wrong formats, such as dates stored as strings.
1. Convert date strings to datetime with pd.to_datetime(df['date']).
2. Convert numerical columns to proper types using df['column'].astype(float).
Example:
Converting a “date” column from string format to datetime allows for easy time-based
analysis like monthly sales trends.

Final Summary:
Data cleaning improves the quality of data and leads to better analysis results. Pandas offers
useful functions like fillna() for missing values, drop_duplicates() for removing duplicates, and
astype()/to_datetime() for fixing data types. These techniques help businesses get accurate
insights from their data and make better decisions.

Questions:
Q1. Explain the importance of data cleaning. List techniques used in Python using pandas.
a) How can pandas be used to handle missing values?
b) What method can be applied to remove duplicates in a dataset?
c) How can pandas help in correcting data formats?
Q2. Why is data cleaning important in analysing sales data? Explain how pandas can help
handle missing values, duplicates, and incorrect data formats.
Q3. Describe the significance of data cleaning in real-world analytics. How does pandas support
cleaning tasks like handling missing data, duplicates, and formatting? Give examples.
Q4. Explain the importance of data cleaning. List three techniques used in Python using pandas.




ANOVA AND ITS TYPES

3. a) What is ANOVA?
ANOVA stands for Analysis of Variance. It is a statistical method used to compare the means
of three or more groups to determine if there are any statistically significant differences
among them. Instead of comparing pairs of groups separately, ANOVA tests all groups
simultaneously by analysing the variances within and between groups. This helps in
understanding whether differences in group means are due to the effect of the factor(s)
being studied or just due to random chance.

b) Key Differences between One-Way ANOVA and Two-Way ANOVA:
 One-Way ANOVA tests the impact of a single categorical independent variable (factor)
on a continuous dependent variable by comparing means across multiple groups.
 Two-Way ANOVA tests the effects of two categorical independent variables (factors) on
a continuous dependent variable and also examines if there is any interaction between
the two factors.
 One-Way ANOVA is simpler and only studies one factor; Two-Way ANOVA studies two
factors and their interaction.
 Two-Way ANOVA provides more detailed insight into how different factors combine
to affect the outcome.

c) Real-World Example of One-Way ANOVA:
A university wants to check if the average exam scores differ by the type of teaching
method used: Traditional, Online, or Hybrid.
Teaching Method Exam Scores (out of 100)
Traditional 78, 82, 85
Online 70, 75, 72
Hybrid 88, 90, 85
Goal: Use One-Way ANOVA to test whether the mean scores differ significantly among the
three teaching methods.
If the test result is significant, it means at least one method leads to different student
performance compared to others.

d) Application and Example of Two-Way ANOVA:

4. A company wants to study the effect of marketing strategy (Email, Social Media) and region
(Urban, Rural) on sales.
Marketing Strategy Region Sales (units)
Email Urban 200, 220
Email Rural 150, 160
Social Media Urban 250, 270
Social Media Rural 180, 190
Two-Way ANOVA helps determine:
1. If marketing strategy affects sales (main effect 1)
2. If region affects sales (main effect 2)
3. If the effect of marketing depends on the region (interaction effect)
For example, social media marketing might work better in urban areas than in rural areas,
showing an interaction.

Final Summary
1. ANOVA is a statistical tool to compare means of multiple groups simultaneously.
2. One-Way ANOVA involves one factor, useful when studying the effect of a single
variable on an outcome.
3. Two-Way ANOVA involves two factors and their interaction, helpful for studying more
complex effects.
4. These methods are widely used in education, business, healthcare, and many research
fields to guide decision-making based on data.
Questions:
Q1.Differentiate between One-Way ANOVA and Two-Way ANOVA with examples.
a) What is ANOVA?
b) Key Differences: How do One-Way ANOVA and Two-Way ANOVA differ?
c) Example: Provide a real-world example of One-Way ANOVA.
d) Application: How is Two-Way ANOVA used to analyze multiple factors?
Q2.Explain the difference between One-Way ANOVA and Two-Way ANOVA. Include a real-life
example of each and describe how Two-Way ANOVA can be used to analyze multiple factors
simultaneously.
Q3.What is the purpose of One-Way ANOVA and Two-Way ANOVA? How do they differ?
Provide an example of when Two-Way ANOVA is preferred over One-Way ANOVA.

5. Data Visualization and Popular Tools
Introduction to Data Visualization?

Data visualization is the practice of converting raw data into visual formats like charts and
graphs to make the information easy to understand and interpret. It helps quickly reveal
trends, patterns, and outliers in the data. For example, a sales manager can view a bar chart
showing monthly sales rather than analysing rows of numbers.

a) Tableau
Usage: Tableau is widely used by businesses to create interactive dashboards and detailed
visual reports without coding. It connects with various data sources like Excel, databases, and
cloud services.

Common Graphs: Bar charts, line charts, heat maps, geographic maps, scatter plots.
Example: A retail company uses Tableau to create a geographic sales heat map showing
regions with the highest product demand during festive seasons, helping them focus
marketing efforts.

b) Power BI
Usage: Power BI is a Microsoft tool used for creating and sharing interactive reports and
dashboards, especially popular in enterprises for real-time data monitoring. It integrates well
with Excel, SQL Server, and cloud data.

Common Graphs: Pie charts, stacked bar charts, waterfall charts, KPI indicators, maps.
Example: A hospital uses Power BI dashboards with pie charts showing patient demographics
and stacked bar charts representing department-wise admission counts to optimize staff
scheduling.

c) Matplotlib
Usage: Matplotlib is a Python library used by data scientists and analysts to programmatically
create detailed static, animated, or interactive plots.

Common Graphs: Line plots, histograms, scatter plots, bar charts, box plots.
Example: A financial analyst plots a line graph of stock prices over 12 months using Matplotlib
to detect seasonal trends and make investment decisions.

6. d) ggplot2
Usage: ggplot2 is an R package used for producing elegant, layered graphics based on a
powerful “grammar of graphics” system. It is heavily used in academic research and statistics.

Common Graphs: Line graphs, scatter plots, histograms, density plots, box plots.
Example: Climate scientists use ggplot2 to plot temperature trends over decades with line
and scatter plots, comparing multiple countries to study global warming effects.

e) Google Data Studio
Usage: Google Data Studio is a free, web-based tool that connects to Google services (Sheets,
Analytics, Ads) to create interactive and real-time dashboards, ideal for marketing and web
analytics.

Common Graphs: Scorecards, time series, geo maps, bar charts, pie charts.
Example: A digital marketing team uses Google Data Studio to display live time series graphs
of website visits and conversion rates, allowing quick adjustments to campaigns.

Summary
Data visualization tools help transform data into meaningful visuals that facilitate faster
understanding and better decisions. The choice of tool depends on the user’s need: Tableau
and Power BI for business intelligence with interactive dashboards; Matplotlib and ggplot2
for detailed scientific and analytical plots; Google Data Studio for real-time marketing
reports. Common graph types like bar charts, line plots, and scatter plots are used across
these tools to effectively communicate data stories.
Questions:
1. Explain the importance of interactive dashboards in business intelligence. Give examples
of tools that support interactive visualization.
2. Write short notes on a) Tableau b) Power BI, c) Matplotlib d)ggplot2 e) Google Data
Studio
3. Briefly describe the purpose and features of any five popular data visualization tools

7. Stationarity in Time-Series Analysis - Important for ARIMA Modeling.
Definition of Stationarity:
Stationarity in time-series analysis means that the statistical properties of a time series —
such as mean, variance, and autocorrelation — remain constant over time. A stationary
series does not have trends, changing variance, or seasonality. This stability allows for reliable
modeling and forecasting.
Types of Stationarity:
1. Strict Stationarity:
The entire probability distribution of the time series remains unchanged over time.
This is a very strong condition and less commonly tested directly.
2. Weak (or Covariance) Stationarity:
Only the first two moments — mean and variance — are constant over time, and the
covariance between two time points depends only on the lag between them, not on
the actual time. This is the most common form used in practice.
Why is Stationarity Important for ARIMA
ARIMA (AutoRegressive Integrated Moving Average) models rely on the assumption that the
input series is stationary. This assumption is important because:
 Stable Mean and Variance: ARIMA uses past data points and errors to forecast future
values. If mean or variance changes over time, these relationships become unreliable.
 Parameter Consistency: Stationarity ensures that model parameters are stable and
valid for prediction.
 Differencing Non-Stationary Data: If the series is non-stationary, ARIMA includes
differencing steps to transform it into a stationary series before fitting the model.
Importance in ARIMA:
 ARIMA models are applied to the stationary differenced data.
 The model can forecast future price changes effectively.
 Forecasted changes can be summed to predict future stock prices.
Real-World Example: Stock Price Forecasting
Consider daily closing prices of a stock over one week:
Day Closing Price ($)
1 100
2 102
3 105

8. 4 107
5 110
6 112
7 115
 This series shows an upward trend; the average price is increasing with time.
 This means the series is non-stationary because the mean changes.

Making the Series Stationary:
 We calculate the difference between each day’s price and the previous day’s price:
Day Price Difference ($)
2 2
3 3
4 2
5 3
6 2
7 3
 The differenced data has a roughly constant mean and variance, indicating it is now
stationary.

Application in ARIMA:
 The ARIMA model will use this differenced (stationary) data to forecast future price
changes.
 By forecasting changes and adding them cumulatively, it predicts future stock prices.

Questions:
Q1.What are the different types of stationarity in time-series data? Explain each with an
example.
Q2.Why is stationarity a crucial assumption for ARIMA modeling in forecasting time-series
data like stock prices?
Q3.What is stationarity in time-series analysis, and why is it important for ARIMA models?
Q4.In stock price forecasting, what is stationarity and why is it important for ARIMA modeling?


Concept of Variability and Describe Standard Deviation, Variance, and Range

9. Variability refers to how spread out or dispersed the data points in a dataset are. It shows
the extent to which values differ from each other and from the mean (average).
Understanding variability helps managers and analysts assess consistency or volatility in data
like sales, production, or test scores.

Measures of Variability:
1. Range:
The difference between the maximum and minimum values in the dataset. It gives a
quick sense of the total spread but is sensitive to extreme values.
2. Variance:
The average of the squared differences between each data point and the mean. It
quantifies how much the data varies overall.
3. Standard Deviation (SD):
The square root of variance, expressed in the same units as the original data. It
indicates typical deviation from the mean and is easier to interpret than variance.

Sales Data Example:
Month Salesperson A (units) Salesperson B (units)
Jan 45 40
Feb 50 60
Mar 55 80
Apr 60 70
May 65 50

Step 1: Calculate Range
 Range for Salesperson A:
Max = 65, Min = 45
Range = 65 - 45 = 20 units
 Range for Salesperson B:
Max = 80, Min = 40
Range = 80 - 40 = 40 units
Interpretation:
Salesperson B has a wider range, indicating more fluctuation in monthly sales compared to
Salesperson A.

10. Step 2: Calculate Mean
 Mean (A) = (45 + 50 + 55 + 60 + 65) / 5 = 275 / 5 = 55 units
 Mean (B) = (40 + 60 + 80 + 70 + 50) / 5 = 300 / 5 = 60 units

Step 3: Calculate Variance
Formula:
Variance = Σ (each value - mean)² / (n - 1), where n = number of data points.
 For Salesperson A:
Month Value Deviation (Value - Mean) Squared Deviation
Jan 45 45 - 55 = -10 (-10)² = 100
Feb 50 50 - 55 = -5 (-5)² = 25
Mar 55 55 - 55 = 0 0
Apr 60 60 - 55 = 5 25
May 65 65 - 55 = 10 100
Sum of squared deviations = 100 + 25 + 0 + 25 + 100 = 250
Variance (A) = 250 / (5 - 1) = 250 / 4 = 62.5
 For Salesperson B:
Month Value Deviation (Value - Mean) Squared Deviation
Jan 40 40 - 60 = -20 400
Feb 60 60 - 60 = 0 0
Mar 80 80 - 60 = 20 400
Apr 70 70 - 60 = 10 100
May 50 50 - 60 = -10 100
Sum of squared deviations = 400 + 0 + 400 + 100 + 100 = 1000
Variance (B) = 1000 / 4 = 250

Step 4: Calculate Standard Deviation (SD)
 SD = √Variance
 Salesperson A: SD = √62.5 ≈ 7.91 units
 Salesperson B: SD = √250 ≈ 15.81 units

11. Interpretation of Variability Measures:
 Range: Salesperson B has a higher range (40 units) than A (20 units), showing B’s
sales vary more widely month-to-month.
 Variance and SD: Both measures confirm that Salesperson B’s monthly sales are more
dispersed (variance 250, SD 15.81) compared to Salesperson A (variance 62.5, SD
7.91).
 Conclusion: Salesperson A has more consistent sales performance, while B
experiences larger fluctuations. A manager might consider A more reliable, or analyze
the reasons behind B's variability for improvement.

Summary:
 Variability helps understand consistency and reliability in data.
 Range shows total spread but is sensitive to outliers.
 Variance measures average squared deviations, indicating overall spread.
 Standard Deviation is a user-friendly measure of typical deviation from the mean.
 These metrics are vital for managers to evaluate sales trends, set targets, and
manage performance.

Questions:
Q1. How does the standard deviation help in understanding the consistency of a salesperson’s
monthly performance? Explain with an example.

Q2. Why is variance squared and how does converting it to standard deviation improve
interpretability in real-world scenarios like customer spending or employee productivity

Q3. Explain the concept of variability and describe standard deviation, variance, and range with
examples.

Q4. A manager wants to compare the monthly sales performance (in units) of two salespersons
over 5 months. The number of units sold by Salesperson A and B are shown below: