Intro to Distributed Systems (By Lasmon Kapota).pptx

It is well presented ppt that is providing an introduction to a concept of distributed systems

1. Intro to Distributed Systems By Lasmon Kapota

2. What is Distributed System? A collection of independent computers that appears to its users as a single coherent system. A system in which components located at networked computers communicate and coordinate their actions only by passing messages. Distributed systems provide a high degree of availability in the face of hardware faults. The availability of a system is a measure of the proportion of time that it is available for use. When one of the components in a distributed system fails, only the work that was using the failed component is affected. A user may move to another computer if the one that they were using fails; a server process can be started on another computer.

3. Characteristics of a Distributed System 1. Composed of autonomous components The system components are able to behave independently of each other. The system components communicate by exchanging messages with each other. 2. Works as a single coherent system The collection of components as a whole operates the same, no matter where, when and how interaction between a user and the system takes place. End users do not even notice that they are dealing with processes, data and control that are dispersed across a computer network. Resource sharing is the main motivating factor for constructing distributed systems.

4. Consequences of the Distributed System Characteristics 1. Concurrency of components The components/nodes can coordinate and execute programs concurrently as they share resources. The capacity of the system to handle shared resources can be increased by adding more resources (e.g. computers) to the network. 2. Lack of global clock There is no single global notion of the correct time. When programs need to cooperate they coordinate their actions by exchanging messages.

5. Consequences of the Distributed System Characteristics Close coordination often depends on a shared idea of the time at which the programs’ actions occur 3. Independent failures of components The failure of a computer, or the unexpected termination of a program somewhere in the system (a crash), is not immediately made known to the other components with which it communicates. Each component of the system can fail independently, leaving the others still running.

6. Examples of a Distributed System Local Area Network and Intranet Database Management System Automatic Teller Machine Network Internet/World-Wide Web Mobile and Ubiquitous Computing

7. Network OS vs Distributed OS Network OS Network OS is a system that connects multiple devices and computers on a network and allows them to share resources on the network. E.g. Microsoft Windows Server, Unix, Linux, Novell Netware, Banyan’s VIVES, etc. Two basic types are peer-to-peer system & client-server system. Primarily concerned with managing resources and communication within a single network. The networked nodes possess their own operating systems.

8. Network OS vs Distributed OS Distributed OS Distributed OS is a system in which two or more systems are connected to each other and work together to serve real-time applications. In other words, it is a system software that manages a network of independent computers and makes it appear as a single system. It uses many central processors to serve multiple real-time applications and users. Connects multiple nodes via a communication channel such as high-speed bus or telephone lines. Types include peer-to-peer system, client-server system, 3-tier, middleware, etc.

9. Network OS vs Distributed OS Network Operating System ( OS) Distributed Operating System ( OS) Main objective is to provide the local services to remote client. Main objective is to manage the hardware resources. Communication is file-based or shared folder-based. Communication is message-oriented and shared memory-oriented Has low fault tolerance. Has high fault tolerance. Easy to implement and maintain. Difficult to implement and maintain. All nodes can have different operating systems. All nodes have the same operating system. Highly scalable. Less scalable. More centralized (Single point of control) Decentralized (Multiple points of control)

10. Middleware A software that bridges the gap between applications and operating systems by providing a method for communication and data management. The capability is useful for applications that otherwise do not have any way to exchange data with other software tools or databases. Its purpose is to mask heterogeneity and to provide a convenient programming model to application programmers to promote interoperability and portability . It is concerned with providing useful building blocks for the construction of software components that can work with one another in a distributed system.

11. Middleware It sits between an OS and applications to connect and share data between the applications.

12. Middleware Components Middleware typically consists of these components: Management console :  Provides an overview of the middleware's activities and configurations. Client interface :  Communicates with the involved applications, databases or other services. Internal interface :  Ensures that components function cohesively. Platform interface :  Ensures compatibility between software. Contract manager :  Defines the procedures that applications must follow for data exchange.

13. Middleware Components Contract manager :  Defines the procedures that applications must follow for data exchange. Session manager :  Creates secure communication channels between the middleware and applications. Database manager :  Integrates with different databases. Runtime monitor :  Continuously monitors middleware data movements and generates reports for developers.

14. Middleware Categories Most middleware falls into one of two categories: Enterprise application integration middleware Enables programmers to create business applications without having to customize integrations for each new application. Provides a f unctionality for data consistency and business to business (B2B) integration so the two software applications can work together. Platform middleware Provides a runtime hosting environment for software development and delivery. M ay provide a container for application program logic, as well as tools like web servers, application servers and content management systems.

15. Types of Middleware There are many types of middleware, each with specific functions and uses. Common types include the following: Messaging middleware , also known as  message-oriented middleware , facilitates communications between distributed applications and services. Object or Object Request Broker (ORB) middleware   enables software components or objects to communicate and interact with a program, such as containers, across distributed systems. Remote Procedure Call (RPC) middleware provides a protocol that enables a program to request a service from another program located on a separate computer or network. Data or database middleware   enables direct access to, and interaction with, databases; it typically includes SQL database software and its own data store.

16. Types of Middleware Transaction or transactional middleware   ensures transactions move from one phase to the next via transaction process monitoring. Content-centric middleware   enables client-side requests for specific content and abstracts and delivers it. Embedded middleware   facilitates communication and integration between embedded apps and real-time operating systems. API middleware   enables developers to create and manage APIs and, in some cases, to monetize APIs. Asynchronous data streaming middleware   enables data sharing between applications.

17. Why Middleware It is used in software development. Enables organizations and developers to build apps more efficiently. Used in data handling, API management and in many authentication & messaging services. It manages connectivity to various back-end resources. Might create a connection pool to provide fast and efficient access to back-end database. Also create connections to message queues & topics and manage connections to cloud-based resources.

18. Why Middleware It implements logic based on client requests. Can take a user request, implement logic and customize the results to meet the client’s preferences. It handles concurrent processing, load balancing and transaction management. Can scale typically vertically and horizontally to distribute incoming client requests over multiple cloud servers. This is called load balancing. Can also handle concurrency and transaction management issues. E.g. problems that arise when multiple clients attempt to simultaneously access or update a back-end resource.

19. Why Middleware It secures access to back-end resources. Usually require both a secure connection using a technology like Secure Sockets Layer and authentication using either a username and password combination or a digital certificate. Sends data to a requesting client only if the client has the authority to access the data. It does so using secure and encrypted connection.

20. Design Issues of a Distributed System Heterogeneity They must be constructed from a variety of different networks, computer hardware, operating systems, programming languages, and implementations by different developers. The differences in programming languages must be addressed if programs written in different languages are to be able to communicate with one another. Programs written by different developers cannot communicate with one another unless they use common standards.

21. Heterogeneity and Middleware A separate layer of software that is logically placed between a higher-level layer consisting of users and applications, and a layer underneath consisting of operating systems and basic communication facilities. A software layer that provides a programming abstraction as well as masking the heterogeneity of the underlying networks, hardware, operating systems and programming languages. The distributed system provides the means for components of a single distributed application to communicate with each other, but also to let different applications communicate. At the same time, it hides, as best and reasonable as possible, the differences in hardware and operating systems from each application.

22. Heterogeneity and Middleware

23. Heterogeneity and Middleware Middleware is the same to a distributed system as what an operating system is to a computer. A manager of resources that offers its applications ability to efficiently share and deploy those resources across a network. Apart from resource management, it offers services that can also be found in most operating systems, including: security services, inter-application communication facilities, accounting services, and masking of and recovery from failures. The main difference with a operating system is that a middleware services are offered in a networked environment.

24. Examples of Typical Middleware Services Communication e.g. remote procedure call. Transactions e.g. atomic transaction. Service composition e.g. Mashups. Reliability e.g. Use of Horus toolkit. Middleware can also be viewed as a container of commonly used components and functions that no longer have to be implemented by applications separately.

25. Heterogeneity and Mobile Code A program code that can be transferred from one computer to another and run at the destination e.g. Java applets. Code suitable for running on one computer is not necessarily suitable for running on another because executable programs are normally specific both to the instruction set and to the host operating system. The virtual machine approach provides a way of making code executable on a variety of host computers: the compiler for a particular language generates code for a virtual machine instead of a particular hardware order code.

26. Heterogeneity and Mobile Code For example, the Java compiler produces code for a Java virtual machine, which executes it by interpretation. The Java virtual machine needs to be implemented once for each type of computer to enable Java programs to run. Today, the most commonly used form of mobile code is the inclusion Javascript programs in some web pages loaded into client browsers.

27. Design Issues of a Distributed System Openness A characteristic that determines whether the system can be extended and reimplemented in various ways. The openness of distributed systems is determined primarily by the degree to which new resource-sharing services can be added and be made available for use by a variety of client programs. May be extended at the hardware level by the addition of computers to the network and at the software level by the introduction of new services and the reimplementation of old ones, enabling application programs to share resources. A further benefit that is often cited for open systems is their independence from individual vendors.

28. Design Issues of a Distributed System Openness To summarize, open distributed systems: Are characterized by the fact that their key interfaces are published. Are based on the provision of a uniform communication mechanism and published interfaces for access to shared resources. Can be constructed from heterogeneous hardware and software, possibly from different vendors. But the conformance of each component to the published standard must be carefully tested and verified if the system is to work correctly.

29. Design Issues of a Distributed System Security Security for information resources has three components: confidentiality (protection against disclosure to unauthorized individuals), integrity (protection against alteration or corruption), and availability (protection against interference with the means to access the resources). Issues like sending sensitive information in a message over a network in a secure manner and identifying a remote user or other agent correctly are met by the use of encryption techniques. However, the following two security challenges have not yet been fully met: denial of service attacks and security of mobile code.

30. Design Issues of a Distributed System Scalability Distributed systems operate effectively and efficiently at many different scales, ranging from a small intranet to the Internet. A system is described as scalable if it will remain effective when there is a significant increase in the number of resources and the number of users. The design of scalable distributed systems presents the following challenges: Controlling the cost of physical resources : As the demand for a resource grows, it should be possible to extend the system, at reasonable cost, to meet it. In general, for a system with n users to be scalable, the quantity of physical resources required to support them should be at most O(n).

31. Design Issues of a Distributed System Scalability Controlling the performance loss : Consider the management of a set of data whose size is proportional to the number of users or resources in the system. Algorithms that use hierarchic structures scale better than those that use linear structures. But even with hierarchic structures an increase in size will result in some loss in performance: the time taken to access hierarchically structured data is O(log n), where n is the size of the set of data. For a system to be scalable, the maximum performance loss should be no worse than this.

32. Design Issues of a Distributed System Scalability Preventing software resources running out : A good example is the adoption of 128-bit Internet addresses as the supply of available 32-bit Internet addresses was running out. This requires modifications to many software components. Avoiding performance bottlenecks : In general, algorithms should be decentralized to avoid having performance bottlenecks. Caching and replication may be used to improve the performance of resources that are very heavily used.

33. Design Issues of a Distributed System Failure Handling Computer systems sometimes fail. When faults occur in hardware or software, programs may produce incorrect results or may stop before they have completed the intended computation. Failures in a distributed system are partial – that is, some components fail while others continue to function. Therefore the handling of failures is particularly difficult. The following techniques are used for dealing with failures:

34. Design Issues of a Distributed System Failure Handling Detecting failures : Some failures can be detected. E.g. checksums can be used to detect corrupted data in a message or a file. However, it is difficult or even impossible to detect some other failures, such as a remote crashed server in the Internet. Masking failures : Some failures that have been detected can be hidden or made less severe. Two examples of hiding failures: Messages can be retransmitted when they fail to arrive. File data can be written to a pair of disks so that if one is corrupted, the other may still be correct.

35. Design Issues of a Distributed System Failure Handling Tolerating failures : Most of the services in the Internet do exhibit failures; it would not be practical for them to attempt to detect and hide all of the failures that might occur in such a large network with so many components. Their clients can be designed to tolerate failures, which generally involves the users tolerating them as well. For example, when a web browser cannot contact a web server, it does not make the user wait for ever while it keeps on trying; it informs the user about the problem, leaving them free to try again later. Recovery from failures : Recovery involves the design of software so that the state of permanent data can be recovered or ‘rolled back’ after a server has crashed. In general, the computations performed by some programs will be incomplete when a fault occurs, and the permanent data that they update (files and other material stored in permanent storage) may not be in a consistent state.

36. Design Issues of a Distributed System Failure Handling Redundancy : Services can be made to tolerate failures by the use of redundant components. Consider the following examples: There should always be at least two different routes between any two routers in the Internet. In the Domain Name System, every name table is replicated in at least two different servers. A database may be replicated in several servers to ensure that the data remains accessible after the failure of any single server. The servers can be designed to detect faults in their peers; when a fault is detected in one server, clients are redirected to the remaining servers.

37. Design Issues of a Distributed System Concurrency Both services and applications provide resources that can be shared by clients in a distributed system. There is therefore a possibility that several clients will attempt to access a shared resource at the same time. The process that manages a shared resource could take one client request at a time. But that approach limits throughput. Therefore services and applications generally allow multiple client requests to be processed concurrently.

38. Design Issues of a Distributed System Concurrency Any object that represents a shared resource in a distributed system must be responsible for ensuring that it operates correctly in a concurrent environment. This applies not only to servers but also to objects in applications. For an object to be safe in a concurrent environment, its operations must be synchronized in such a way that its data remains consistent. This can be achieved by standard techniques such as semaphores, which are used in most operating systems.

39. Design Issues of a Distributed System Transparency The concealment from the user and the application programmer of the separation of components in a distributed system, so that the system is perceived as a whole rather than as a collection of independent components. Transparency hides and renders anonymous the resources that are not of direct relevance to the task in hand for users and application programmers. For example, it is generally desirable for similar hardware resources to be allocated interchangeably to perform a task; the identity of a processor used to execute a process is generally hidden from the user and remains anonymous.

40. Design Issues of a Distributed System Eight Forms of Transparency Access transparency enables local and remote resources to be accessed using identical operations. Location transparency enables resources to be accessed without knowledge of their physical or network location (e.g. which building or IP address). Concurrency transparency enables several processes to operate concurrently using shared resources without interference between them. Replication transparency enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers.

41. Design Issues of a Distributed System Eight Forms of Transparency Failure transparency enables the concealment of faults, allowing users and application programs to complete their tasks despite the failure of hardware or software components. Mobility transparency allows the movement of resources and clients within a system without affecting the operation of users or programs. Performance transparency allows the system to be reconfigured to improve performance as loads vary. Scaling transparency allows the system and applications to expand in scale without change to the system structure or the application algorithms.

42. Design Issues of a Distributed System Transparency The two most important transparencies are access and location transparency; their presence or absence most strongly affects the utilization of distributed resources. They are sometimes referred to together as network transparency.

43. Design Issues of a Distributed System Quality of Service Once users are provided with the functionality that they require of a service, such as the file service in a distributed system, they then think about the quality of the service provided. The main nonfunctional properties of systems that affect the quality of the service experienced by clients and users are reliability, security and performance. Adaptability to meet changing system configurations and resource availability has been recognized as a further important aspect of service quality. The performance aspect of quality of service was originally defined in terms of responsiveness and computational throughput, but it has been redefined in terms of ability to meet timeliness guarantees.

44. Recap on Key Characteristics of a Distributed System Heterogeneity Openness Security Scalability Failure Handling Concurrency Transparency Quality of service

45. GOOD LUCK!!! By Lasmon Kapota