"Grid technologies for distributed computing and data processing" curriculum

for training directions (specialties) :
“6.050101 – Computer sciences", 7.080402 "Information technologies of design"


І. GENERAL INFORMATION

1.1 The  topic of educational discipline

Discipline "Grid technologies for distributed computing and data processing " belongs to the cycle of disciplines of professional and practical training and is based on knowledge of disciplines : "The algorithmic programming fundamentals", "Object-oriented programming", "Computer networks".

Total number of credits of ЕТС8 - 4,5,  the number according to the list of disciplines of the training program - NP-08

1.2 A place of educational discipline in the system of professional knowledge.

This educational discipline belongs to the cycle of the professionally-oriented disciplines. The study of this discipline requires the students to have basic knowledge on generally-theoretical and generally-engineering disciplines. The complex of knowledge formed by this discipline belongs to principles of professional training, and the quality of specialists’ training depends on it. With his knowledge and skills acquired a specialist will be able to use grid technologies to solve complex scientific - practical problems.

Supporting disciplines:  "Introduction to information technologies", "Data structures and algorithms", "Programming and algorithmic languages".


 

ІІ. TIME SCHEDULE

 

Semester

Credit. module coding

Total cred./ hours

By the types of classes

(all hours/hours in a week)

Individual work

Module tests

(quantity)

Individual. task

(type)

Semester

certification

Lectures

Practical/

seminar

Laboratory/

Computer practical work

10

NP-08

4,5/162

36/2

-

18/1

108

1

-

exam


 

ІІІ. DISCIPLINE GOALS AND TASKS

This discipline is aimed at study of basics of grid technologies used to support scientific research, and acquisition of practical skills of the use of grid technologies to solve scientific - practical tasks. In a result of study of this discipline students must:
to know: grid technologies fundamentals, that allow to unite computing resources and data storage resources as the single system; architecture of the grid systems used in Ukraine; principles of functioning of basic grid components; technology of grid tasks preparation;
be able: to choose and to use middleware to solve scientific - practical tasks; to adapt applications to the grid environment; to use input languages for job and data description; to monitor and to correct the execution of tasks during a computing experiment or simulation process.
gain skills of: practical use of the most widespread middleware; work at the access portal.
An educational process of the discipline consists of lectures, laboratory works and individual work.


 

ІV.1. TIME SCHEDULE BY TOPICS

 

Name of sections, topics

Time distribution by semesters and types of classes

Total

Lectures

Practical.

Seminar.

Laboratory.

Computer practice

Individual

Semester 10

SECTION 1. Grid concept

1.1 Introduction to grid technologies

6

2

4

1.2 Grid architecture. Grid computing resources

6

2

4

1.3 Internet as grid component

6

2

4

1.4 Grid security

6

2

2

2

SECTION 2. Grid middleware

2.1 General structure of middleware

6

2

4

2.2 Grid user work session scenario

6

2

4

2.3 Structure and components of ARC middleware

16

4

4

8

2.4 Structure and components of gLite middleware

14

4

4

6

2.5 Providing the interoperability of grid systems

8

2

6

2.6 Grid information system. Monitoring.

12

2

4

6

2.7 Resource scheduling in grid.

10

2

2

6

2.8 Portals and meta-schedulers.

8

2

6

2.9 Grid service development

8

2

6

SECTION 3 New directions of grid technologies

3.1 Grid for data processing

10

2

8

3.2 Semantic grid

10

2

8

3.3 Cloud computing

8

2

6

Module test (sections 1 and 2)

10

2

8

Preparation for exam

10

10

Total

162

36

18

108


 

IV.2. LECTURES

SECTION 1. Grid concept

Topic 1.1 Introduction to grid  technologies
Lecture 1. Introduction to grid  technologies

1.    Goals, tasks and contents of the course.
2.    Grid concept
3.    History of grid development.
4.    Examples of grid applications.
Tasks for ind.work:
1.    Evolution of grid (first, second and third generations of grid).
2.    International grid projects (GrossGrid, EGEE, Grid2003, myGrid)
[L1, p. 5-10, p. 55- 132],[L2, p. 10-17],[L5, p. 1-145].

Topic 1.2 Grid architecture. Grid computing resources
Lecture 2.  Grid computing resources

1.    Basic components of grid architecture.
2.    Computing resources.
3.    Clusters.
4.    Data storage resources
Tasks for ind.work:
1.    Basic trends of computing engineering development.
[L4, p. 1-214], [L2, p. 28-41].

Topic 1.3 Internet as grid component. Ukrainian national grid.
Lecture 3. Internet as grid component

1.  Basic directions of development. Cyber-infrastructure.
2.  GEANT2 project.
3.  Development of communication channels in Ukraine.
4.  Grid in Ukraine.
5.  Ukrainian national Grid – the project of UNGI for EGI .
Tasks for ind.work:
1.  Development of communication channels in Russia.
[L2, p. 18-25].

Topic 1.4 Grid security
Lecture 4. Grid security

1.  Features of grid services security model.
2.  General principles of cipher algorithms.
3.  Authentication of users and grid resources.
4.  X.509 certificate
5.  Virtual organizations
6.  User work with a certificate scenario
Tasks for ind.work:
1.  Delegation.
2.  Virtual organizations management services.
[L2, p. 79-81].

SECTION 2.  Grid middleware

Topic 2.1 General structure of middleware.
Lecture 5. General structure of middleware.

1.  Basic grid services.
2.  Job management subsystem.
3.  Data management subsystem.
4.  Information subsystem and monitoring subsystem.
5.  Security and access subsystem.
6.  Logging subsystem
Tasks for ind.work:
1.  Resource utilization accounting subsystem.
2.  Organization of data storage resources.
[L1, p. 21-28], [L2, p. 54-78]

Topic 2.2 Grid user work session scenario.
Lecture 6. Grid user work session scenario.

1.  Basic functions of user interface.
2.  Acquiring the certificate.
3.  Job definition.
4.   Grid job submission commands.
5.  Job execution monitoring.
6. Retrieving the job results
Tasks for ind.work:
1.  Types of grid jobs.
2.  Methodology of job preparation on Linux cluster.
3.  Local Resource Management System / PBS.
[L5, p. 10 - 121]

Topic 2.3 Structure and components of ARC middleware
Lecture 7. Structure and components of ARC middleware

1.  NorduGrid Project.
2.  Architecture of ARC (Advanced Resource Connector).
3.  Information system.
4.  Architecture and working algorithm of ARC grid manager.
Tasks for ind.work:
1.  Architecture and working algorithm of  GridFTP.
[L2, p. 82-84], [L10, L11, L12, L13, L14, L15, L16, L17, L18]

Lecture 8. Structure and components of ARC middleware (continuation)
1.  Job execution in ARC.
2.  Basic commands of ARC user interface.
3.  xRSL language of job definition.
Tasks for ind.work:
1.  Monitoring in ARC.
[L10, L11, L12, L13, L14, L15, L16, L17, L18]

Topic 2.4 Structure and components of gLite middleware
Lecture 9. Structure and components of gLite middleware

1.  LCG and EGEE projects.
2.  gLite architecture.
3.  Workload Management System.
4.  Data Management System.
5.  Information System.
6.  Logging and Bookkeeping.
Tasks for ind.work:
1.  Accounting Subsystem.
[L19, L20, L21, L22]

Lecture 10. Structure and components of gLite middleware (continuation)
1.  Job execution in gLite
2.  Basic commands of gLite user interface.
3.  JDL language of job definition.
Tasks for ind.work:
1.  Types of jobs : Collection, DAG, Parametric.
2.   Monitoring in gLite.
[L19, L20, L21, L22]

Topic 2.5 Providing the interoperability of grid systems.
Lecture 11. Providing the interoperability of grid systems

1.  The interoperability problems of grid systems
2.  Basic models of interoperability provision.
3. Providing the interoperability between ARC and gLite MW
4.  Development and introduction of grid standards.
Tasks for ind.work:
1.  Grid Interoperability Now Project (GIN).
2.  Providing of grid interoperability within the framework of EGGE project.
[L5, p. 1-145]

Topic 2.6 Grid information system. Monitoring.
Lecture 12. Grid information system. Monitoring

1.    Tasks of the information and monitoring system.
2.    Architecture of the monitoring systems.
3.    Modern monitoring systems of grid environments.
4.    Interaction with the information system.
Tasks for ind.work:
1.  Architecture of the Globus MDS 2/4, MonALISA, GridICE, GridMaps monitoring systems.
[L10, p. 120-134]

Topic  2.7 Resource scheduling in grid.
Lecture 13. Resource scheduling in grid.

1.  Tasks of resource scheduling in grid.
2.  Resource scheduling implementation technologies.
3.  Methodology of grid resource load estimation.
4. Resource scheduling in gLite and ARC MW.
Tasks for ind.work:
1.  Grid meta-schedulers.
2.  Optimal grid resource load balancing algorithms.
[L3, p. 9-89, p. 309-356], [L24 ]

Topic 2.8 Portals and meta-schedulers.
Lecture 14. Portals and meta-schedulers

1.  Architecture of grid portals.
2.  Examples of portals.
3.  GILDA grid access portal.
4.  Architecture of grid meta-schedulers.
5.  Examples of grid meta-schedulers.
Tasks for ind.work:
1.  Work with GILDA portal.
2.  A knowledge portal for Ukrainian grid infrastructure.
3.  SD Grid portal.
[L2, p. 84-102]

Topic 2.9 Grid service development
Lecture 15. Grid service development

1.  Introduction to Service Oriented Architecture (SOA).
2.  Basic grid service development phases.
3.  Examples grid service development.
Tasks for ind.work:
1. Globus Toolkit
[L24 ], [L6 of p. 1-273 ]

SECTION 3  New directions of grid technologies

Topic 3.1 Grid for data processing
Lecture 16. Grid for data processing

1.  Features of grid data processing systems architecture.
2.  Basic tasks.
3.  Examples of implementation.
4.  Data Mining systems: an intelligent data processing.
5.  Data Mining system classes.
Tasks for ind.work:
1. LHC Project .
2. WizWhy as an example of data mining system
[L7, p. 1-286], [L8, p. 1-336] [L9, p. 1-384]

Topic 3.2 Semantic  grid
Lecture 17. Semantic  grid

1.  Basic tasks of semantic grid.
2.  Basic components of semantic grid architecture.
3.  Languages for ontology description.
4.  Examples of implementation.
Tasks for ind.work:
1. Ontology description language OWL.
2. Architecture of S - OGSA
[L9, p. 1-384]

Topic 3.3 Cloud computing
Lecture 18. Cloud computing

1.  Basic tasks of Cloud computing architecture.
2.  Examples of Cloud computing architectures.
3.  Architecture of Amazon Web Services.
Tasks for ind.work:
1. Tasks and architecture of GrepTheWeb system.
2. Hadoop software
[L38]


 

ІV.3. LABORATORY WORK

A primary objective for laboratory classes is to practically consolidate the knowledge given at lectures and gain sufficient practical skills of work with the modern computer systems. These practical skills, as well as personal computer applications experience, are gained during individual work on the PC.

Lab #1 Grid security -  4 hours
Lab #2 Advanced Resource Connector (ARC) middleware -  8 hours
Lab #3 gLite middleware -  8 hours
Lab #4 Grid information system and monitoring -  8 hours
Lab #5 Grid resource scheduling -  8 hours
Lab #6 Module test №1 (sections 1 and 2)  -  2 hours


 

ІV.4. INDIVIDUAL TASKS

Topics for individual work and topics for the deeper study of the material are listed in table IV.


 

ІV.5. TESTING

Module test is conducted for the sections 2 and 3. It is these sections that contain the most difficult topics for students. Additional individual work of students and corresponding control are therefore needed.


 

V. METHODOLOGICAL GUIDANCE

Lectures and labs are the base methods of studies  (each student has his own variant of lab task). Materials for laboratory classes (corresponding methodological guidance) at the System design department disposal.


 

VІ. TEACHING MATERIALS

 

Literature & references

1.  Петренко А.И., Застосування Grid технологій в науці та освіті: роздатковий матеріал до вивч. курсу для студ. спец. «Інформаційні технології проектування» - К.: НТУУ «КПІ», 2008,- 144 .  ( http://moodle.ntu-kpi.kiev.ua)
2.  Петренко А.И., Вступ до  Grid технологій в науці та освіті: навчальний посібник. - К.: НТУУ «КПІ», 2008,- 120 с.( http://moodle.ntu-kpi.kiev.ua)
3.  Пономаренко В.С., Листровой С.В., Минухин С.В., Знахур С.В., Методы и модели планирования ресурсрв в Grid системах. – Х.:ВД. «ІНЖЕК», 2008.- 408 с.  
4.  Introduction to Grid Computing, December 2005, -IBM Redbook, www.ibm.com/redbooks - 241 c.
5.  Grid Computing in Research and Education, April 2005, - IBM Redbook, www.ibm.com/redbooks - 145 c.  6.  Grid Services Programming and Application Enablement, May 2004,- IBM Redbook www. ibm.com/redbooks, - 273 c.
7 .  Паклин Н.Б., Орешков В.И., Бизнес-аналитика: от данных к знаниям (+ СD), Издательский дом Питер, 1-е издание, - 2009 год,- 624 с.
8.  А.А. Барсегян, М.С. Куприянов, В.В. Степаненко, И.И. Холод.  Методы и модели анализа данных: OLAP и Data Mining (+ CD-ROM). Издательство: БХВ-Петербург, -2004 г.- 336 с.
9. А. А. Барсегян, М. С. Куприянов, В. В. Степаненко, И. И. Холод,  Технологии анализа данных. Data Mining, Visual Mining, Text Mining, OLAP (+ CD-ROM). - Издательство: БХВ-Петербург, - 2007 г. - 384 с.
10. NorduGrid project. http://www.nordugrid.org
11. The NorduGrid Grid Manager And GridFTP Server: Description And Administrator’s Manual. http://www.nordugrid.org/papers.html
12. The NorduGrid Brokering Algorithm, M.Ellert, http://www.nordugrid.org/papers.html
13. xRSL (Extended Resource Specification Language), O.Smirnova,  http://www.nordugrid.org/papers.html
14. Usage statistics and usage patterns on the NorduGrid, K.Pajchel,
http://www.nordugrid.org/papers.html
15. ARC User Interface: User’s Manual http://www.nordugrid.org/documents/NorduGrid-UI.pdf
16. The NorduGrid ”Smart” Storage Element, A.Konstantinov, http://www.nordugrid.org/papers.html
17. The NorduGrid/ARC Information System, (Technical Description and Reference Manual),  Bal´azs K´onya,  http://www.nordugrid.org/papers.html
18. The Grid Monitor: Usage Manual, http://www.nordugrid.org/documents/monitor.pdf
19. GLITE 3.1 USER GUIDE , https://edms.cern.ch/file/722398/1.2/gLite-3-UserGuide.pdf
20. Logging and Bookkeeping, A. Kˇrenek et al., ttp://egee.cesnet.cz/cvsweb/LB/LBAG.pdf.
21. EGEE User’s Guide, WMS SERVICE , https://edms.cern.ch/document/572489/1
22. JDL Attributes Specification, EGEE-JRA1-TEC-555796-JDL-Attributes-v0-6 , https://edms.cern.ch/file/555796/1/.
23. The Resource Broker Info file, DataGrid-01-TEN-0135-0_0 , http://www.infn.it/workload-grid/docs/DataGrid-01-TEN-0135-0_0.doc.
24. Web Services (http://www.w3.org/2002/ws/)

Additional materials

25.    Grid Computing Making the Global Infrastructure a Reality,  edited by Fran Berman,
Geoffrey Fox, Tony Hey. – (Wiley series in communications networking & distributed systems), 2003 , 1007 с.
26.   Portal Application Development Using WebSphere Portlet Factory, IBM Redbook www. ibm.com/redbooks January 2008, 697 c.
27. Openldap. http://www.openldap.org
28. Monitoring and Discovery Services. http://www.globus.org/mds/mds2/
29. Runtime Environment Registry, http://www.csc.fi/grid/rer/
30. The GLUE Information model versin 1.2 http://infnforge.cnaf.infn.it/glueinfomodel/
31. EGEE Middleware Architecture, DJRA1.1 ,https://edms.cern.ch/document/476451/1.0.
32. Global Security Architecture, DJRA1.3 ,https://edms.cern.ch/document/487004/1.1.
33. VOMS User’s Guide, EGEE-JRA1-TEC-571991 ,https://edms.cern.ch/file/571991/1/voms-guide.pdf.
34. www.datagrid.org
35. www.unicore.org
36. www.glite.org
37. www.globus.org
38. http://book.itep.ru/4/7/cloud.htm
39. BDII homepage, https://twiki.cern.ch/twiki/bin/view/EGEE/BDIIv4
40. Петренко А.І., Булах Б.В.,Хондар В.Д. Семантичні грід- технології для науки і освіти:додатковий матеріал. -// К.: НТУУ «КПІ», 2010.- 178 c.  ( http://moodle.ntu-kpi.kiev.ua)


Curriculum is developed on the basis of 6.080402,  “Computer sciences” program
Author                                     Svistunov S.Y.
Head of the department      prof. Petrenko A.I.

 

 

Ukrainian (UA)Russian (CIS)English (United Kingdom)