Banesco Basico DataStage DIA 01-1 chapter 1pdf

chap[ter 1

1. © 2019 IBM Corporation
BÁSICO DE PARALLEL JOBS
DIA 01

2. © 2019 IBM Corporation2
Agenda DataStageDia 01
•Introduccióna Information Server y DataStage.
•Topologías.
•Arquitecturadel Job Parallel
•Fuentes de consultas: Knowledge Center y Manuales.
•Introducciónal DataStage Designer
•CreaciónParallel Jobs
•Stages: RowGenerator, ColumnGenerator, Peek, Head y Tail.
Parte 2

3. © 2019 IBM Corporation
IBMINFOSPHERE
INFORMATIONSERVER

4. © 2019 IBM Corporation4

5. © 2019 IBM Corporation
What is IBM InfoSphere DataStage?

6. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Features

7. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Features

8. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Pre Build Stages

9. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Pre Build Stages

10. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Components

11. © 2019 IBM Corporation
What is IBM InfoSphere DataStage? Types Jobs

12. © 2019 IBM Corporation
Installation Overview

13. © 2019 IBM Corporation
Installation Overview -Tiers

14. © 2019 IBM Corporation
Installation Overview –Client Tier

15. © 2019 IBM Corporation
Installation Overview –Services Tier

16. © 2019 IBM Corporation
Installation Overview –MetaData Tier

17. © 2019 IBM Corporation17
ARQUITECTURA PARALLEL JOBS

18. © 2019 IBM Corporation18
Arquitectura en Paralelo

19. © 2019 IBM Corporation19
DataStage documentación job tipo Parallel

20. © 2019 IBM Corporation20
Conceptos claves de paralelismo

21. © 2019 IBM Corporation21
Scalables Ambientes de Hardware

22. © 2019 IBM Corporation22
Procesamiento tradicional en Batch

23. © 2019 IBM Corporation23
Paralelismo de Tuberias

24. © 2019 IBM Corporation24
Paralelismo de Particionamiento

25. © 2019 IBM Corporation25
Ilustración de Particionamiento

26. © 2019 IBM Corporation26
DataStage Combina Paralelismo y Pipelining

27. © 2019 IBM Corporation27
Diseño de Jobs versus Ejecución

28. © 2019 IBM Corporation28
Definiendo el Paralelismo

29. © 2019 IBM Corporation29
Archivo de Configuracion

30. © 2019 IBM Corporation30
Archivo de Configuracion mostrado en el job log

31. © 2019 IBM Corporation
KNOWLEDGE CENTER

32. © 2019 IBM Corporation32
https://www.ibm.com/support/knowledgecenter/en/SSZJPZ_11.3.0/com.ibm.swg.im.iis.produ
ctization.iisinfsv.relinfo.doc/topics/iisihrinfo_infsv_rnote_v113.html

33. © 2019 IBM Corporation33

34. © 2019 IBM Corporation34

35. © 2019 IBM Corporation35
Manuales del Information Server
https://www.ibm.com/support/pages/infosphere-information-server-version-113-product-documentation

36. © 2019 IBM Corporation36

37. © 2019 IBM Corporation37
INTRODUCCION AL
DATASTAGEDESIGNER

38. © 2019 IBM Corporation38
Creación Parallel Job

39. © 2019 IBM Corporation39
Arrastrar Stages y Links

40. © 2019 IBM Corporation40
Renombrar Stages y Links

41. © 2019 IBM Corporation41
Vista desde el Director

42. © 2019 IBM Corporation42
Job Log visto desde el Designer

43. © 2019 IBM Corporation43
CREACIONDE PARALLELJOBS

44. © 2019 IBM Corporation44
¿Qué es un Parallel Job?

45. © 2019 IBM Corporation45
Stage Row Generator

46. © 2019 IBM Corporation46
Stage Column Generator
•Generatemockdata forspecificcolumnsforeachdata row
processed
•Single input link ; Single Output link
•OnPropertiesTab, specifyhowtheColumnGeneratorstage
operates: Explicit, SchemaFile.
•Algorithmsforintegertype
✓Random; Seed, Limit
✓Cycle: Initalvalue, Increment
•AlgorithmsforStringtype: Cycle, alphabet
•AlgorithmsforDate type: Random, cycle

47. © 2019 IBM Corporation47
Stage Peek

48. © 2019 IBM Corporation48
Stage Head
•Selects the first Nrows from each partition of an input data sets
and copies the selected rows to an output data set
•Single input link ; Single Output link
•OnPropertiesTab, specify:
✓The number of rows to copy
✓The partition from which the rows are copied
✓The location of the rows to copy
✓The number of rows to skip before the copying operation begins
•This stage is helpful in testing and debugging applications with
large data sets. For example,
✓The Partition property lets you see data from a single partition to
determine if the data is being partitioned as you want it to be.
✓The Skip property lets you access a certain portion of a data set.

49. © 2019 IBM Corporation49
Stage Tail
•The Tail Stage selects the last N records from each partition of an
input data set and copies the selected records to an output data
set
•Single input link ; Single Output link
•OnPropertiesTab, specify:
✓The number of records to copy
✓The partition from which the records are copied
•This stage is helpful in testing and debugging applications with
large data sets. For example,
✓The Partition property lets you see data from a single partition to
determine if the data is being partitioned as you want it to be.
✓The Skip property lets you access a certain portion of a data set.

50. © 2019 IBM Corporation50
Agenda DataStage
BÁSICO DE PARALLEL JOBS
•Importaciónde Metadata
•Accediendo data Secuencial.
•Stages: Secuencial File, Data Set, Copy.
•Accediendo data No Estructurada.
•Stage: UnstructuredData.
•Particionamiento, colección y archivo de Configuración.
•Combinando Datos
•Stages: Lookup, Merge, Join, Funnel
Parte 1

51. © 2019 IBM Corporation
IMPORTACION DE METADATA

52. © 2019 IBM Corporation52
SourceandTargetMetadata

53. © 2019 IBM Corporation53
Sequencialfile importprocedure

54. © 2019 IBM Corporation54
Importingsequencialmetadata

55. © 2019 IBM Corporation55
Sequencialimportwindow

56. © 2019 IBM Corporation56
Specifyformat

57. © 2019 IBM Corporation57
Editcolumnsnamesandtypes

58. © 2019 IBM Corporation58
ExtendedPropertiesWindows

59. © 2019 IBM Corporation59
Tabledefinitionintherepository

60. © 2019 IBM Corporation
ACCEDIENDO DATA SECUENCIAL
SEQUENTIALFILE STAGE

61. © 2019 IBM Corporation61
Cómo se maneja la data sequencial

62. © 2019 IBM Corporation62
Caracteristicas del Stage Sequential File

63. © 2019 IBM Corporation63
Ejemplo de Formato Sequential File

64. © 2019 IBM Corporation64
Diseño de Job con stage Sequential File

65. © 2019 IBM Corporation65
Propiedades Stage Sequential File

66. © 2019 IBM Corporation66
Tab Format

67. © 2019 IBM Corporation67
Tab Columns

68. © 2019 IBM Corporation68
Múltiples Lectores

69. © 2019 IBM Corporation69
Escribiendo en un Sequential File

70. © 2019 IBM Corporation70
Links de Rechazos (reject)

71. © 2019 IBM Corporation71
Links de Rechazos

72. © 2019 IBM Corporation72
Fuentes y Destinos links de Rechazdos

73. © 2019 IBM Corporation73
Ajuste Propiedad Modo de Rechazo

74. © 2019 IBM Corporation74
ACCEDIENDO DATA SECUENCIAL
Copystage

75. © 2019 IBM Corporation75
Copy Stage

76. © 2019 IBM Corporation76
Ejemplo Copy Stage

77. © 2019 IBM Corporation77
Tab Mapping

78. © 2019 IBM Corporation78
ACCEDIENDO DATA SECUENCIAL
Data Set stage

79. © 2019 IBM Corporation79
Data Set Stage

80. © 2019 IBM Corporation80
Job con un Data Set de destino

81. © 2019 IBM Corporation81
Utility Data Set Management

82. © 2019 IBM Corporation82
Visualizar data y Schema

83. © 2019 IBM Corporation
ACCEDIENDO DATA NO ESTRUCTURADA
UNSTRUCTURED DATA STAGE

84. © 2019 IBM Corporation84
UnStructured File Stage
•Unstructureddataisinformationthatdoesnothaveapredefined
datamodelordoesnotfitwellintorelationaltables.Unstructured
datacanbetextfrombooks,journals,metadata,audio,video
files,thebodyofwordprocessordocuments,webpages,and
presentationcharts
•Oneinput link
•MultipleOutput link (largeamountofmemory)
✓Onlysupportformat(.xlsx)
✓MaximunnumberofExcel recordsis1.048.576
•Currentlysupportsonlyreadand writestoExcel sheets:
✓Excel 97-2003 (.xls)
✓Excel 2007-2010 (.xlsx)
•Supports password-encrypted files
•Doesn´t support Excel files created on Mac

85. © 2019 IBM Corporation85
Visualizar data y Schema, Lectura hoja Excel

86. © 2019 IBM Corporation86
Primera Hoja del Libro , no se procesa

87. © 2019 IBM Corporation87
Segunda Hoja a Procesar

88. © 2019 IBM Corporation88
Propiedades

89. © 2019 IBM Corporation89
Configurar

90. © 2019 IBM Corporation90
Resultado de la Extracción

91. © 2019 IBM Corporation91
Grabación data y Schema, grabación en hoja Excel

92. © 2019 IBM Corporation92
Visualizar data de entrada

93. © 2019 IBM Corporation93
Visualizar columnas

94. © 2019 IBM Corporation94
Propiedades

95. © 2019 IBM Corporation95
Configurar

96. © 2019 IBM Corporation96
Visualizar

97. © 2019 IBM Corporation
PARTICIONAMIENTO,
COLECCIÓN Y
ARCHIVO DE CONFIGURACIÓN

98. © 2019 IBM Corporation98
Paralelismo de Partición

99. © 2019 IBM Corporation99
Stage partitioning

100. © 2019 IBM Corporation100
Algoritmos de Particionamiento

101. © 2019 IBM Corporation101
Collecting

102. © 2019 IBM Corporation102
Collecting

103. © 2019 IBM Corporation103
Algoritmos de Coleccionamiento

104. © 2019 IBM Corporation104
Algoritmos de Particionamiento

105. © 2019 IBM Corporation105
KEYLESS vs KEYED Algoritmos de Particionamiento

106. © 2019 IBM Corporation106
Particionamiento Round Robin y Random

107. © 2019 IBM Corporation107
Particionamiento Completo (KEYLESS)

108. © 2019 IBM Corporation108
Particionamiento HASH (KEYED)

109. © 2019 IBM Corporation109
Particionamiento HASH (KEYED)

110. © 2019 IBM Corporation110
Particionamiento MODULUS (KEYED)

111. © 2019 IBM Corporation111
Particionamiento Auto

112. © 2019 IBM Corporation112
Iconos Partitioning / Collecting

113. © 2019 IBM Corporation113
Otros Iconos Partitioning / Collecting

114. © 2019 IBM Corporation114
Archivo de Configuración

115. © 2019 IBM Corporation115
Uso del Archivo de Configuración

116. © 2019 IBM Corporation116
Ejemplo Archivo de Configuración

117. © 2019 IBM Corporation117
Editando Archivos de Configuración

118. © 2019 IBM Corporation
COMBINANDO DATOS
STAGELOOKUP

119. © 2019 IBM Corporation119
Combinando datos

120. © 2019 IBM Corporation120
Características Stage Lookup

121. © 2019 IBM Corporation121
Ejemplo Stage Lookup búsqueda exacta

122. © 2019 IBM Corporation122
Stage Lookup con búsqueda exacta

123. © 2019 IBM Corporation123
Definiendo clave para la búsqueda (Lookup)

124. © 2019 IBM Corporation124
Especificando columnas de salida

125. © 2019 IBM Corporation125
Acciones en falla de busqueda

126. © 2019 IBM Corporation126
Especificando acciones en falla de busqueda

127. © 2019 IBM Corporation127
Búsqueda Con Link De Rechazo

128. © 2019 IBM Corporation128
Funcionamiento Stage Lookup

129. © 2019 IBM Corporation129
Fallas Stage Lookup (ejemplos)

130. © 2019 IBM Corporation
COMBINANDO DATOS
STAGEJOIN

131. © 2019 IBM Corporation131
Stage Join

132. © 2019 IBM Corporation132
Stage Join ejemplo

133. © 2019 IBM Corporation133
Propiedades Stage Join

134. © 2019 IBM Corporation134
Tab Mapping Salidas

135. © 2019 IBM Corporation135
Funcionamiento Stage Lookup

136. © 2019 IBM Corporation136
Salida Inner Join

137. © 2019 IBM Corporation137
Salida Left Outer Join

138. © 2019 IBM Corporation138
Salida Right Outer Join

139. © 2019 IBM Corporation139
Salida Full Outer Jooin

140. © 2019 IBM Corporation
COMBINANDO DATOS
STAGEMERGE

141. © 2019 IBM Corporation141
Stage Merge

142. © 2019 IBM Corporation142
Ejemplo Stage Merge

143. © 2019 IBM Corporation143
Propiedades Stage Merge

144. © 2019 IBM Corporation144
Comparasion entre Join, Lookup y Merge

145. © 2019 IBM Corporation
COMBINANDO DATOS
STAGEFUNNEL

146. © 2019 IBM Corporation146
¿Qué es un stage Funnel ?

147. © 2019 IBM Corporation147
Ejemplo Stage Funnel

148. © 2019 IBM Corporation148
Propiedades Funnel Stage

149. © 2019 IBM Corporation149
Gracias.!