National Grid-infrastructure for scientific research Development

Project № ІТ /506 - 2007 from August 22, 2007

Ugrid Project (www.grid.kpi.ua) deals with the development of the MESU(MONU) segment of National Grid-infrastructure with relevant services for granting an opportunity for native scientists to cooperate effectively with the European Research Area, ERA.
Initiation of the information society economics that is based on knowledge with implementation scientific Grid concepts and with the most influential scientific applications, which is used in Grid-environment. Ugrid project was developed by a temporary research team which consisted members of following native organizations:

• National Technical University of Ukraine "Kyiv Polytechnic Institute" (KPI) 
• Institute for Modeling Problems in Energetics G.E. Pukhova National Academy of Sciences of Ukraine (IPM) 
• Kharkiv National University of Radio Electronics (KNURE) 
• Lviv National Technical University "Lviv Polytechnic" (NULP) 
• Zaporizhzhya National Technical University (ZNTU) 
• Donetsk National Polytechnic Institute (DonNPI) 
• Dnipropetrovsk National Mining University (DNHU) 
• Enterprise USTAR, 
• State Enterprise "Lviv Radio Engineering Research Institute 
(LNDRI) 

During two years of Ugrid project development based on research of modern condition and progress tendencies of existing solutions, embedded in European and Global projects of Grid-environment, an educational segment of national Grid-infrastructure was build. It consists basic educational resource-operating center at the NTUU “KPI”, where the Supercomputing Center with a powerful cluster (www.hpcc.org.ua) ,World Data Center “Geoinformatics and Sustainable Development)(www.wdc.org.ua) as well as 5 regional educational resource-operating centers in Kharkiv, Donetsk, Dnipropetrovsk, Zaporizhzhya and Lviv are operating. 

Basic Research Results:

Ugrid Project implementation was carried out on a timetable that has 4 stages.
The following paragraphs include major achievements during each stage. 
The first development phase (УІ.2007 - ІХ.2007):
- Analysis of the concept, architecture and resources of modern Grid-systems were performed, information on methods of building architecture with distributed systems based on Grid technologies systematized.

- Grid-infrastructure of MONU segment was built with 6 educational resource-operating centers (in Kyiv, Kharkiv, Donetsk , Dnipropetrovsk, Zaporizhzhya, Lviv) and remote service for users-scientists from Ukrainian universities and scientific institutions was established. Of course this initial Grid-infrastructure can be developed and number of its centers and developers from MONU organizations and scientific institutions NANY will increase. The fractality of Grid-infrastructure supports this enhancement, particularly its feature when system is similar to itself on every scale-different section.

An agreement with European organization DANTE concerning the connection of National Research and Academic Network URAN (www.uran.net.ua) to the European network GEANT-2 was signed in June 2007.
URAN Network (URAN - Ukrainian Research and Academic Network) was created by the Ministry of Education and Science of Ukraine (MESU) and by the National Academy of Sciences of Ukraine (NASU) to provide Ukrainian institutions, organizations and individuals in education, science and culture with informational services, based on Internet-technologies for implementation of their professional needs and for development of these industries. The main management center is located in Kyiv, and regional centers are located in the largest scientific centers of Ukraine-- Kharkiv, Dnipropetrovsk, Donetsk, Odessa, Lvov, Simferopol, Khmelnitskyy.
1

URAN network 

The second phase ( Х.2007 – ХІІ.2007)
-Comparative analysis of the components of Grid-middleware packages was conducted and on its base was recommended to use ПГ3 gLite in generating Grid-environment, which has a wide range of services, supports service-oriented architecture, is widely used in European projects, and has sufficient capacity for modernization and creation of additional functionality. The main approaches to the problem of interoperability of existing PHZ were considered.

-three-language information site of the project was developed (www.grid.ntu-kpi.kiev.ua) Its information resources is now exceeded 80MB. Similar sites were created in regional resource centers, for example, in Kharkiv (www.grid.kture.kharkov.ua); 
- the agreement with EUGridPMA about the implementation of Certificate Center in Ukraine, that is responsible for Grid-resource, users’ and virtual organizations’ registration, has been reached.
-the agreement with NASU has been reached and integration of existing segments of scientific and educational computing and communication NAA and AAA infrastructure into a single Ukrainian National Grid-Initiative (UNGI) has started. UNGI is integrated into European Grid Infrastructure EGEE (Enabling Grids for E-sciencE) and EGI (European Grid Iniriatives).

The Third phase ( УІ..2008 – ІХ.2008)

-Clusters ETP NASU and NTUU “KPI” (ІТФ НАНУ і НТУУ «КПІ») were connected based on ПГ3 NorduGrid for analysis of the Virtualization Technology of national computational resources with simultaneous connections to cluster NTU KPI and 5 educational resource - operational centers. A joint program on Grid-technology was approved with Korean universities through KOICA organization in the Ukrainian-Korean Center for IT, which implementation has already begun;
- In January 2008 the Public Key Certification Center launched to provide Ukrainian users with access to European resources and data storages (www.ca.ugrid.org).
Today this center supports users of both segments of Grid-infrastructure (academic and educational)
• The access portal SDGrid (www.sdgrid.org.ua) to the Grid-infrastructure was developed as user’s single access point to heterogeneous information resources and applications that will be involved in the resource - operational centers for service and training. 
Grid portal is a software and hardware, with the main features of--- assignment formulation 
for computations in Grid-infrastructure, monitoring resources and security policy in Grid-infrastructure, users training, providing access to another Grid-systems, developing virtual organization etc.  
It is Based on Gridsphere with applications for support ПГ3 gLite, Globus 2 / 4, and Gridsphere provides developer with an extremely efficient mechanism to create new applications that are
using the portlet system (Fig. 2)

2
Fig.2. Page of Grid-portal

The fourth phase ( Х.2008 -ХІІ.2008):
Methodology of using Grid-technology in higher school was studied and an experimental educational program on Grid-course was created; a master training program :Grid-technologies in Science and Education” (Politechnika 2008) and “Application of Grid-technology in science” (Politechnika 2009) were established; a special joint team from MESU and NASU (MONU and NANU) has began a preparation of a manual ; Grid- technology masters and specialists designing was organized and the best results are presented on this website in the “personnel training for grid-technology” section.

-a virtual Grid-educational system was created on the basis of the developed SDGrid portal; with its help one can get a first hand experience in Grid-environment. Also on the basis of this portal a new virtual laboratory was developed, it can be used for demonstration all the features of Grid-technology, and include- 5 geographically distributed resources: Web-server, MyProxy, server, CA (Certificate Authority), HPC computing resources server, monitoring server and backup server for collecting status statistics from server. Thanks to this virtual laboratory you can get the first experience in Grid-environments.  
This experience includes registration and MyProxy certification, reviewing files in the directory, navigation within the directory, assignment definition and execution, tracking running state etc.
• Grid-segment MESU(MONU) (unlike the NAS Grid-segment, which is a computing Grid) can be attributed to the information type Grid (Data Grid), because the project Ugrid is mainly associated with service granting for Ukrainian World Data Center "Geoinformatics and Sustainable Development" (UWDC), by providing its customers with remote access to global storages of scientific data, with the possibility of efficient combined usage of computers, unique experimental installations and devices (www.wdc.org.ua). 
UWDC is part of World Data Center (Fig. 3) which is established and supported by International Council for Science. It collects, stores and exchanges data with other centers in the following areas: sustainable development and globalization research, solid state science, solar-terrestrial physics and other 

4Computational recourses. The official funding of the Ugrid project begun only in the second half of 2007, operations on its execution had begun just after the approval of the State Program. Also in October 2006 an Inauguration of the Center for High Performance Computing and UWDC was held. Earlier in April 2006, partnership with the executor of the European project BalticGrid was established. 
Today after the third moderation NTUU “KPI” cluster’s power is 5, 83 5, 83 TFLOPS \ s, ie 5.83 billion operations per second. It has 44 nodes with Two four-core processors Intel Xeon E5440 (2.83HHts and 8 GB RAM each) and 78 nodes with two dual-core processors Intel Xeon 5160 (3.00HHts and 4 GB of RAM each) ie with 624 computing cores in general . InfiniBand for data exchange and Linux CentOS release 5.2. were used.
NTUU “KPI” has a difference- additional system with 128 cores, ie with 16 nodes with two four-core processors Intel Xeon E5345 (2.33 GHz, 8 GB of RAM and 500 GB disk each), which is running MS Windows Server 2008 HPC Edition. Thereby first time in Ukraine users, who are used to Windows OS, are granted the opportunity to apply parallel computing through the parallelization program MS MPI 2.0.1551. 
Cluster of the Cybernetics Institute- SCIT-3 with its 5, 32 TFLOPS \s is inferior to ours.
But it is not important for national users which cluster is more efficient- 
IC NASU or NTU "KPI", because they both are integrated into a single Grid-network, where assignments are automatically divided by resource manager, considering their size and availability of computing resources.

This resource virtualization is one of the advantages of Grid-built infrastructure. 
Today users are provided with a parallelization system OpenMPI 1.2.8, with compilers C + +: intel 10.1, gcc 4.1.2,П3 Application: type GROMACS 4.0.2 (molecular dynamics analyzer), GEMESS (crystal structures analyzer ), fftw 3.2 (subprogram library for calculating the discrete Fourier transform) 
NTUU “KPI” cluster’s inclusion into the grid-infrastructure allows its clients to use ПП3 type MATHEMATIКA and GAUSSIAN (program complex for determining molecular orbital density), which was purchased by the Institute of Theoretical physics (ITP) for academic segment in Grid-infrastructure

Scientific Research. All organizations who participate in the project (except organizational duties of resource-operating center, operation guarantee, user training and support) conducted researches in middleware interoperability of different Grid-infrastructure, in information protection while the association of national Grid-infrastructures or their segments; in additional applications development for expanding the range of possible users (besides science) for members of various sectors of society: engineering, business, social sphere. Additional researches that were conducted are following:

• NTU "KPI" –knowledge portal creating, interoperability research, middleware research, Grid-based software for modern micro-electronic mechanical systems (MEMS) development; 
• NTU Lviv Polytechnic –powerful data storage system creating using IBM BladeCenter QS21, which consists of 14 computing blade servers based on Cell processors and one coordinating node; 
• ZNTU (Zaporizhzhya National Technical University) – research of parallel structures of genetic algorithms with the help of parallel multilayer neural networks and with usage of the cluster for direct construction of neural network, development and implementation of the parallel replica-method algorithm, that is based on modulating of molecular-dynamic approach for surface thermosetting diffusion in a range of a second of simulated time
• KNURE (Kharkiv National University of Radio Electronics) - Research of Grid usage for development of distributed systems, simulation models, developing software for performance testing of library PVM; 
• ISPE (Institute of simulation problems in energetics) – research of methods of using Grid-technologies for solving technological problems in energetics, 

Methods research of effective information interchange between system nodes in HPC-cluster of distributed access, methods research of reconfigurable computing techniques in Grid-computing, 
numerical design of interconnected electromagnetic, thermal and hydrodynamic processes in electromagnetic systems by example on analysis of process dynamics of the interoperation of equilibrium air plasma with a metal electrode (plasma system), development of the mathematical model and program complex that allow to explore process of heat-mass transfer in technological channels of RBMK-1000 reactor in different work modes and in the time of destruction, as well as in “Shelter” and others.

The following phases are:
To provide researches with possibility of using Grid resources during the computations in familiar environments that are installed on personal computers; on the NTUU “KPI” cluster, NetSolve MATLAB / Mathematica / Fortran / C software was installed. Now user does not have to care where the required Grid-resource is located, how it is detected and activated; he only needs to specify the criteria that have to be matched by this resource, and further he interacts with this resource in the same way as with the local resources (procedures, classes, programs) of its operating environment

While realizing that issue of filling of the national Grid-infrastructure with application programs for parallel computing in various fields of modern science and engineering are getting more significant in Ukraine- Ugrid projects developers plan to pay special attention to the next phase of the implementation of the State Program IKT. They consider involving development of semantic Web-portal of knowledge for access to national and global information resources.

According to the experimentations more than 36 scientific papers and theses on various conferences were published, and one of the most remarkable was the 21 conference CODATA 2008: “SCIENTIFIC INFORMATION FOR SOCIETY – FROM TODAY TO THE FUTURE”, covering the usage of scientific data in modern society and future society.
.
Several sections of this conference covered the sizing up of research on Grid – projects, conducted by NASU(NANU) and MESU(MONU); Dr. Robert Jones from CERN, a director of the world largest Grid-EGEE (Enabling Grids for E-science - Grid- infrastructure for e-Science ,was present on those reports, he highly appreciated the state of Grid-development in Ukraine.

The main principles of the New State Program in Grid-technology implementation and usage in Ukraine in 2009-2013 were approved; For the First time the National Academy of science of Ukraine (NANU) and The Ministry of Education and Science were initiators of this program.

Ukrainian (UA)Russian (CIS)English (United Kingdom)