Scientific and medical expertise in quality provision system of scientific researches in health sphere (Methodical recommendations) (стр. 1 )

MINISTRY OF HEALTH OF

THE REPUBLIC OF KAZAKHSTAN

RSE «INSTITUTE OF HEALTH DEVELOPMENT»

SCIENTIFIC AND MEDICAL EXPERTISE IN QUALITY

PROVISION SYSTEM OF SCIENTIFIC RESEARCHES IN

HEALTH SPHERE

Methodical recommendations

Astana, 2011

UDC

BBC

Scientific and medical expertise in quality provision system of scientific researches in health sphere. – Methodical recommendations. – Astana. – 2011. -_p.

BBC

These methodical recommendations are intended for use by Ministry of Health and organizations of medical science of the Republic of Kazakhstan. Use of proposed reglament and principles of scientific and medical expertise will give a possibility to conduct independent estimation of scientific production – projects and reports of scientific and technical programs on applied scientific researches in health sphere.

Approved and recommended by Republic Center of Innovational technologies of Medical education and science of the Ministry of Health of the Republic of Kazakhstan. (protocol from «__»________________№ ____).

(С) 2011

List of abbreviations

SSTE	state scientific and technical expertise
HEI	higher education institution
SRC SRW	scientific and research center scientific and research work
SC	scientific center
STP	scientific and technical program
GCP	Good Clinical Practice
GLP GCLP QPBR	Good Laboratory Practice Good Clinical Laboratory Practice Quality Practice in Biomedical Research

Содержание

INTRODUCTION…..…...……………………………………………..	5
Scientific and medical expertise in scientific researches quality provision system in health sphere
1.  Domestic experience analysis of scientific and technical expertise
2. International experience analysis of scientific and research works estimation
3. International experience analysis of scientific and medical expertise criteria elaboration
4.  Scientific and medical expertise procedure in health system of the Republic of Kazakhstan
4.1.  Scientific and medical expertise procedure of scientific and technical programs and budget applications projects
4.2. Scientific and medical expertise procedure of scientific and technical programs and budget applications projects
CONCLUSION…………………………………………………...
TABLE OF AUTHORITIES ……………...
APPLICATIONS …………………………………………………..

Scientific and medical expertise in scientific researches quality provision system in health sphere

1.  Domestic experience analysis of scientific and technical expertise

Competitive elaboration appearance of domestic scientists is necessary condition in way interpretation of Kazakh medical science in international research space. Quality development of scientific researches in health sphere was identified as one of important and priority tasks of branch strategical development by Ministry of Health of the Republic of Kazakhstan.

Accepting necessity of active measures in quality control sphere of scientific researches is noticed in Conception of medical science reformation of the Republic of Kazakhstan for , are confirmed by order of Ministry of Health of the Republic of Kazakhstan № 79 from 19th of February 2008. Continuousness education of scientific personnel of scientific research methodology conduction is realizing under realization of conception for last years in accordance with international standards and requirements, is elaborated and implemented the effective estimation instruments of science activity organization. One of key arrangements which require further implementation in domestic medical science system is the conduction of independent scientific and medical expertise.

Acceptance of Codex of the Republic of Kazakhstan in 2009 “About nation health and health system” based legal basics for scientific and medical expertise establishment in domestic health system. List of objects was identified, which are liable to scientific and medical expertise: projects of fundamental and applied scientific researches programs; republican purposeful scientific and medical programs; results of completed scientific and medical programs; scientific works, nominating on search of state awards of the Republic of Kazakhstan; scientific and medical elaborations, planning for implementation in health practice.

Scientific and medical expertise procedure is identified by authorized agency – Ministry of Health of RK and must be based on international estimation criteria of scientific researches quality.

Absence of exact regulation and any other instructive documents in scientific and medical expertise sphere in domestic health system identifies high actuality of researches conduction and elaborations in this sphere.

Absolutely, in mechanisms base of quality estimation branch system of scientific researches and in particularly, using principles at state scientific and technical expertise conduction (SSTE). SSTE is a system of evaluative actions and conclusions are organized by central executive body, who realizes management in science and scientific and technical activity sphere – by Ministry of Education and Science of The Republic of Kazakhstan – for receiving of independent analytical estimation of expertise object – projects and reports of scientific and technical programs [1].

In base of SSTE conduction is underlining such principles as independence and objectiveness; competence and high professional persons and organizations rate, which conducts expertise; orientation on world rate of science and techniques development, compliance of technological and ecological safety norms and rules, standard requirements;

Sistemacy of expert work organization and it normative and scientific and methodic provision; fullness and authenticity of using and generate information; responsibility for authenticity, fullness and validity of expert recommendations and complexity, consists in estimation of scientific and technical, economic, ecological and social sequences of expert object realization; public openness [2].

SSTE is based on expert estimation of scientific researches on such parameters as general actuality for science development, conformity of receiving results to assigned research tasks, conformity to needs of social and economic development of the Republic of Kazakhstan, novelty, scientific and practical significance of receiving results SRW, researches difference on this project from early conducting (conducted) SRW in RK and other foreign countries, approbation level of receiving results (scientific publications, reports on conferences, pre-patents, patents and others) and it readiness to realization, export and import substitution possibilities, competitiveness of received scientific and technical production on internal and external market [2].

2. International experience analysis of scientific and research works estimation

International experience analysis shows the presence of specialized criteria, which use for scientific research estimation as on stage of project planning, and on stage of reports presentation on completed researches.

Swedish Research Council is the direct government agency, which response for quality provision of scientific researches as key criteria at quality estimation of scientific projects requiring financing with part of state uses following criteria [3]:

Scientific quality "Scientific excellence) – an indicator, which characterize:

- Conception reasonableness and quality of assigned purposes and tasks,

- possibility advance in new achievement;

- quality and effectiveness, chosen research methodology and conformity to it research plan;

Project implementation "Quality and efficiency of the implementation and the management" – an indicator which characterize:

- relevance of planning arrangements,

- quality and experience of individual research participators,

- enough of dedicated resources (budget, personnel, equipment).

Impact “Potential impact through the development, dissemination and use of project results”) – an indicator which characterize:

- project contribution, as on National and European level;

- measure appropriateness for extension and/or usage of project results and intellectual property management.

Every criterion estimates on five-point scale.

Estimation shows what rate of every criteria achievement of every criterion in this project:

0 – Application on project cannot be estimated on this criterion from absence or insufficiency presented information

1 – Very bad. Criterion shows in surface and unacceptable manner.

2 – Bad. It has serious integral weak point relatively concerned criteria.

3 – Enough. In spite on project application opens broadly this criterion; it has essential weaknesses, which need in correction.

4 – Good. Project application shows good this criterion, although is possible another identified modernizations.

5 - Excellent. Project application successfully shows all proper aspects of concerned criterion. Any limitations are insignificant.

Applications on projects are presented by universities, scientific and research institutes and other public research organizations in accordance with specified succession criteria.

In addition to that experts estimate every project under accordance criterion in full form, but are not some separate subcriteria. Subcriteria – questions which expert must consider in the estimation of this criterion.

Estimation procedure includes following steps:

1. First step includes the succession self-certification of every application. Every application must satisfy to appropriate national succession criteria.

2. Second step involves in the individual application estimation by experts/representatives, authorized Council.

3. Third step – collegial applications consideration on base of individual estimations and decision making about financial discharge.

Effective instruments of applications estimation on scientific projects are used by European commission, which organize competitions under Seventh framework programs FP7 [4,5]. In addition these independent experts realize an estimation of represented projects on three basic criteria:

Science and technology criterion – including the estimation:

- soundedness of concept and quality of objectives – this indicator estimates the project idea

- novelty (Progress beyond state-of-the-art) – on this indicator is necessary to demonstrate the knowledge of that has (data of literature survey), and say what will done new;

- quality and effectiveness of proposed scientific/technological methodology and plan of work – this indicator shows an estimation of purposes separation on tasks and subtasks, showing that researcher knows how he/she will decide this task, which state.

Criteria of project implementation possibilities (Implementation) – including the estimation:

- adequacy of project management procedures and structure;

- adequacy of every participator experience and level;

- consortium quality as whole group (mutual addition and balance of participator competence spheres);

- adequacy of resource allocation (persons, equipment) and it substantiation.

Criteria of project significance (Impact), including the estimation:

- summary contribution to expected impact, which shows in work project program;

- Measure adequacy, which accept for dissemination of project results and questions account of intellectual property.

Every criterion estimates from 0 till 5 balls with possibility of separate subcriteria estimation. Independent experts appraise applications according to recommendations of European commission on united schema, issue from set criteria. For application conduction on every from criterion must be gained not less 3 balls from 5 possible, but sum estimation must not less 10 balls (maximal ball - 15). On every criterion is possible the estimation with usage of half balls (i. e. 3, 5, 4, and 5). Application consideration can be stop if one of criteria has 2 or less balls. Application estimation conducts closely and distantly, person who gives application to know only about final decision of experts.

In addition, specificity of research conduction in health sphere, including questions of ethical regulation and possibility of further elaborations use in medical practice, as well as necessity of specialized scientific research standards compliance in health sphere (GLP, GCP, GCLP, QPBR and others) requires inclusions in expertise process of specific criteria and indicators.

3. International experience analysis of scientific and medical expertise criteria elaboration

Elaboration of effective criteria for scientific and medical expertise conduction is an important term for scientific and reasonable principles implementation of administrative decision approval by authorized bodies in health sphere.

Search of international experience indicates about specific instruments presence and specified conventional criteria list of scientific researches quality estimation in health sphere.

Criteria of scientific and research projects estimation are financed under grant program, which is organized by affiliative centers (National Institute of Cancer and others) concerns:

Significance Criteria Significance is based on project estimation with following positions:

- How much the important problem is planning to search?

- if purposes are achieved, how it will promote to development of scientific knowledge and clinical practices;

- as research results will influence on conception, methods, technologies, medication methods, services and prophylactic arrangements, which use in this medicine sphere.

Estimation Criteria of methodological approach are based on project estimation with positions:

- how much it adequacy is elaborated, integrated and motivated conceptual and clinical structure, design, methods and analysis in relations of project purposes and tasks;

- how much fully project applicant recognized a potential problematic area and considered all possible alternatives.

Innovation Criteria are based on project estimation with positions:

- how much project is original and innovational, i. e. the example, how much project challenges to real paradigms and clinical practice, if it directs innovational hypothesis on development of definite area.

- whether it supposes the elaboration project of new conceptions, approaches, methodologies, instruments and technologies in studying health area.

Estimation Criteria of Investigators are based on project estimation with positions:

- how much researches are correct prepared for this research execution;

- how much required project is concerning to experience level of researches, who implicate in project;

Estimation Criterion of Project Environment is based on project estimation with positions:

- how much scientific environment contributes the possibility of successful results achievement;

- what take advantage of required research from project scientific environment and population is involve in the project, or use possible cooperation ways;

- have whether confirmation of institutional support.

According to instruction “Estimation of planning and completed scientific and research and experience and constructive elaborations”, which is elaborated by Ministry of Health of the Republic of Belarus (5th of December 2006, Registration № ) was proposed a number estimation criteria of planning and completed scientific and research and experience and constructive works (STP) in system of Ministry of Health of the Republic of Belarus [6].

Planning SPT is estimated on three directions: characteristic of theme idea SPT (criteria “novelty”, characteristics of criteria expected result “significance for science and practice”), characteristics of singers and probability of positive problem decision (criteria “objectivity”).

Criteria “Novelty” shows a presence of new scientific knowledge scientific activity in expected results, which can characterize within the scope of well-known (novelty is absent, theme idea is generally known – estimates in “0” mark) till high (research conducts for the first time in World, is planning to receive the competitive scientific result, which conformity to world novelty – estimates in 7-10 marks).

Intermediate significance is taken by criteria indicators “relatively high scientific novelty” – characterizes planning themes SIP, which is executed for the first time in the country (estimates in 4-6 marks) and “insufficiency high scientific novelty”, which characterize researches, novelty of which will on rate of organization-executor (estimates in 1-3 marks).

Criteria “significance for science and practice” opens through following indicators:

Indicator “influence level of expected result of science and practice”, reflective level of expected scientific result influence on science, economics, social sphere, health practice, which can be: high – effects on further science and practice development (estimates in 7-10 marks), relatively high – effects on further science and practice development (estimates in 4-6 marks), insufficiency high (estimates in 1-3 marks), significance is absent – will not have significance for science and practice development (estimates in “0” marks).

Indicator “actuality of planning STP” is showing inferred degree of idea conformity of planning STP to needs of science and practical health. This indicator can be high, if it has paramount importance for science and practice (estimates in 7-10 marks), relatively high, if research has an important meaning for further development of science and practice (estimates from 4 till 6 marks), insufficiency high, if research has a definite meaning for science and practice (estimates in 1-3 marks) or irrelevant, if the planning research doesn’t conform to inquiries and tasks of science and practice (estimates in “0”mark).

Indicator “Rate estimation of expected effect from implementation” is showing in medical and prophylactic, social, economic and scientific effects.

Medical and prophylactic effects are characterized by methods improvement of prophylactics, diagnostics and treatment diseases and expressed in morbidity decrease, medication period decrease, decrease of disability, mortality and others.

Social effect is characterized by scientific elaborations influence level on public health and expressed in increase of life time, natural population increase, life promotion, and increase of quality life, improvement of labour conditions, environmental protection and quality increase of medical service.

Economic effect shows decrease of charges at rendering of medical aid and increase of public labour production, including the receiving of direct economy from scientific elaborations adoption in medicine and health, release of more effective medical means and medical techniques, increase of import substitution and export of technologies and elaborations.

Scientific effect is characterized by growth level of new scientific and medical information (knowledge), received during elaboration execution.

Every kind of effect estimated within the range form 1 till 10 marks in depends from estimated level, significance and scale of implementation. Effect absence from implementation, or indication is not planning, estimates in 0 marks.

Under presence of some kinds of effect in sum the effect of supposed implementation mustn’t increase 10 marks. The indicator “Rate estimation of expected effect from implementation” must be excepted from analysis and from sum of marks for fundamental indicator SRW.

Criteria “Objectivity” opens through following indicators:

Indicator “methodic level and material and technical base of planning research”, is characterized by scientific validity level of expected results: methodic level estimation of supposed research and material and technical base of executor (with gradation “high”, “relatively high”, “insufficiency high”, “low”, estimating from 10 till 0 marks).

Indicator “Basic executor’s qualification” differentiates executors on research theme on relevant and irrelevant level of planning research (yes, no).

Indicator “Presence of backlog” shows the presence of positive results, which confirm the correctness of chosen direction, publications presence on research theme and others and estimates with gradation from 10 till 0 marks.

Indicator “validity of execution period SPT” shows the execution period validity of planning SPT (period is understated, substantiated, overstated). The indicator is estimated quality (yes, no) and if it possible, with indication of % understating or overstating.

Indicator “financial validity STP” shows a financial validity level of planning theme STP (cost is understated, substantiated, overstated). The indicator is estimated quality (yes, no) and, if it is possible, with indication % understating and overstating.

Estimation of planning STP on first stage realizes Scientific Councils of organizations-executors. After receiving of positive conclusion from Academic Council on execution place of planning STP is conducting the expertise in one of expert commission at Academic medical council MH RK. Every expert of expert commission makes out marks for every of considering STP.

Thereafter data on every criterion is summed up and averaged, that permits to level the subjectivism of separate experts on these or other criteria.

Themes of STP are recommended to planning, which gained high or satisfactory sum of marks on all estimated criteria.

Estimation of completed STP realizes in marks on following criteria: criteria “novelty”, criteria “significance for science and practice”, criteria “validity and exactness”.

Criteria “Novelty” shows presence in receiving results of new scientific knowledge (new scientific information), technologies novelty, which can be characterized on novelty level on following gradations: “high” – research results have world novelty, not analogues in world science and practice, presence of patents and innovations (estimated in 7-10 marks), “relatively high” – novelty on republic level, presence of rational proposals (estimates in 4-6 marks), low – novelty conforms to rate of separate SRO (estimates in 1-3 marks).

Novelty level is characterized by such results, as elaboration of normative and legal, normative and technical and instructive and methodic documents, which confirms on republic rate or branch ministry instead of innovations (patents) and rational proposals in themes of STP.

Criteria “significance for science and practice” opens through following indicators:

Indicator “rate of scientific significance”. The indicator shows the influence rate of receiving scientific result on development of science, scientific directions and scientific schools, which can be: “high”- cardinally influence on further development of science development in research sphere (estimates in 7-10 marks), “relatively high”- substantially influence on further development of science in research theme sphere (estimates in 4-6 marks), “low” – effects on some positive influence on science development in research sphere (estimates in 1-3 marks). In estimating of completed STP on this criteria pay attention to publications presence in foreign printing, appearances of scientific forums and others.

Indicator “significance for practice”. The indicator shows the influence degree (scale) of receiving scientific result on practice: health, social sphere, economics, which can be: “high”- cardinally influence on practice in health sphere (estimates in 7-10 marks), “relatively high” – substantially influence on practice in health sphere (estimates in 4-6 marks), “low” – exert some positive influence on practice in health sphere (estimates in 1-3 marks).

Indicator “Implementation rate and volumes”. The indicator shows implementation scale and level of results STP, presence and number of economic, social, medical and prophylactic effect in national economy or in health practice. This criterion can be “high” – widespread adoption in republic with significant economic, social, medical and prophylactic effect (estimates in 7-10 marks), “relatively high” – implementation in some organizations with social, medical and prophylactic effect (estimates in 4-6 marks), “low”- implementation in one organization without significant social, medical and prophylactic effect (estimates in 1-3 marks). In estimating of completed STP on this criterion pay attention the having acts of implementation, participation in international and republic exhibitions, serial or experimental production of drugs, equipments and medical-purpose products, based on results of conducting researches and others.

Criterion “validity and exactness”. This criterion opens through following indicators:

Indicator “validity level of results”. This indicator characterizes scientific validity of received research results: statistical validity and repeatability of research results, used methods validity and reliability of static handling, presence of comparative groups, adequate controls and randomization, methodic rate and research documentation, clinical researches conduction within the scope of evidentiary medicine. In spite of results evidentiary rate can change within the rage from indefiniteness till reproduction and application on practice, in the estimating of completed STP on this criterion the estimation realizes only on quality indicator “conform to accepted standards”.

Indicator “conformity to plan STP”. The indicator characterizes positions conformity, including in report STP and received results on calendar plan and plan tasks STP. It is estimating on two parameters: “conform to plan STP” or (not conform to plan STP).

Estimation of completed (intermediate) STP on this mark scale on first stage is realized by Academic councils of organizations-executors. After positive conclusion receiving of Academic Council on work execution place of every completed STP which conduct the expertise in one of expert commissions under Academic medical council of MH RB. Experts makes out marks on every of considering STP. Thereafter data is summed on every criterion and averaged that permits to level the subjectivism of separate experts on these or other criteria. The completed STP can be accepted, which received in high and satisfy marks sum on all evaluation criteria, as well as positive marks received on criterion “validity and exactness of conducted researches” and on criterion “conformity to plan STP”.

4.  Scientific and medical expertise procedure in health system of the Republic of Kazakhstan

According to Codex of the Republic of Kazakhstan from 18.09.2009 N 193-4 “About population health and health system” covers objects of scientific and medical expertise:

1) projects of fundamental and applied scientific researches programs;

2) republican purposeful scientific and research programs;

3) results of completed scientific and medical programs;

4) scientific works, nominating for search of state awards of the Republic of Kazakhstan;

5) scientific and medical elaborations, planning for implementation in health practice.

International experience of scientific and scientific and medical expertise conduction shows on usage necessity of following principles at estimation conduction of scientific and technical programs reports and projects – independence, objectivity; competency and high professional level of persons and organizations, which conduct an expertise [1-13].

Independence of expertise conduction and high level of expert professionalism must provide through base creation of independent experts. Basic criteria at selection of independent experts must be the professional and scientific preparation level, experience of professional, scientific and expert activity, personal qualities and science contribution.

Level of professional and scientific preparation must be estimated on presence of high/post-graduate medical education, presence of academic degree, presence of additional education (professional development, seminars, master-classes) in management sphere of scientific and research activity.

Professional, scientific and expert activity experience can be estimated on presence of work experience in health system, presence of participation experience in scientific and technical programs execution on applied scientific researches in health sphere, experience presence in expert estimation conduction, scientific works and dissertation researches reviewing

Personal quality and science contribution can be estimated on presence of scientific achievements (titles, awards, scholarships, appreciations, and honour certificates), presence of scientific works (publications, patents).

Independent experts’ base must form from number of leading scientists and specialists’ health organizations, conforming to stated criteria. The programs implementation is necessary for regular qualification raising and continuousness of independent experts’ professional development.

Natural organizational and technical maintenance of scientific and medical expertise must realize the work body. According to work specificity Institute of health development can appearance.

During scientific and medical expertise is necessary to use an exact algorithm of scientific production estimation, in addition must be used selective approach to different objects of scientific and medical expertise – projects STP and reports on execution results SRW.