Agendas for the 3rd National Research and Development Agency Council （Draft）
Since measures for supporting Research and Development agency-originated ventures were enhanced and their activities were added by the enactment of the Act on Activation of the Creation of Science and Technology Innovation, it is necessary to consider how to appropriately do and evaluate each agency’s activities. In particular, since the funding is financed by each agency’s self-income, outreach activities are increasingly required Therefore, we are planning to examine successful cases on these activities.
National Research and Development Agencies have large-scale research infrastructure （equipment and databases of its research facilities） based on the mission shown in the mid to long-term objective. We are planning to examine successful cases of the use of research infrastructure taking advantage of these characteristics of each agency.
We are planning to follow up amendments in each agency based on the results of the survey conducted in FY2016 on successful cases of management by the President of each agency. Moreover, we are planning to examine the management by the President of each agency by carrying out anonymous written questionnaire surveys for its management-level officials.
Some projects require a certain period of time to gain results because they have to construct facilities or their themes are far from social implementation. We are planning to identify how to effectively operate and evaluate them so that we get common suggestions to mid to long-term R&D projects.
We are planning to survey common matters to National Research and Development agencies such as initiatives for getting and fostering human resources and that for collaboration with related organizations with the aim of utilizing its results for reviewing the organization and business of each agency.
As five years have passed from the establishment of the National Research and Development Agency system, various systemic bottlenecks may have arisen. We are planning to identify such issues and get a suggestion toward improvements of system operations necessary for each agency to maximize research outcomes.
○ Towards improvement of the effectiveness of the evaluation from a viewpoint of “maximization of research achievements，” it is important to evaluate the degree or speed of attainment which is related to the outcome of researches．For the evaluation，attention should be paid to medium to long-term objectives shown by a flowchart（visualized flow from initiatives to outputs，outcomes and impacts）．
○ Each agency should provide a road map showing how to implement medium to long-term objectives／plans，and clarify how its initiatives are advancing．
○ When preparing a flowchart and a roadmap，their contents should be as specific as possible in order to support the validity of the evaluation．Achievements which were not intended in original objectives and not described in a flowchart or a roadmap should also be evaluated justly．
○ When conducting the evaluation，not only achievements but also issues should be presented so as to contribute to solving the issues．
○ Upon evaluation in each sub-committee of National Research and Development Agency Council，basic stances including that toward the judgment standard of evaluation should be clarified by each item and shared by sub-committee members who are involved in the evaluation．
○ When each sub-committee implements the evaluation，it is necessary to note whether excessive differences are generated in judgment of each sub-committee，from the viewpoint of balance of weighting of respective evaluation item（for example，the item of maximization of R&D achievements，that of appropriate and effective business operation，and that of efficiency improvement of business operation）in the entire evaluation of an agency．
○ A view capable of implementing evaluation of the management of an agency including management of R&D and cooperation with other organizations is required．It is favorable to establish a certain directionality by making draft common evaluation items of management．
○ When self-evaluation is made by a national research and development agency or a party to be evaluated makes an explanation，they should accumulate grounds which are as objective and specific as possible，such as indexes established for measuring effects of their efforts as well as results of such efforts．
○ When evaluating achievements applicable to a plurality of evaluation items，basically，each item should be evaluated by a different viewpoint so that the same achievement is not evaluated redundantly．At the same time，cautions are also required not to underestimate the achievements due to distribution and subdivision of each item of achievements．
○ Other than relative evaluations to determine whether achievements of the agency are excellent or not in light of global standard，absolute evaluations based on the standard which can be assumed considering potential level of the agency have been implemented．It is required to clarify that，in light of the relation to outcomes／outputs included in medium to long-term objectives and the characters of business operations，which type of evaluation should be implemented preferentially．Regarding this issue，some views should be investigated and improved by all the evaluators．
○ Even for R&D with high risk where it is difficult to anticipate a certain level of achievement within a short-term period，an appropriate evaluation for such a given agency should be made in accordance with its own traits．It is advised that such an evaluation should not retard challenging and long-term R&D．
○ Unexpected R&D achievements irrelevant from the original objective may be made． Although it is difficult to evaluate such achievements under the condition of the current objective-oriented evaluation methods，such achievements should be proactively highly evaluated as well．
○ Regarding R&D being conducted collaboratively by multiple agencies，even in case some issues may arise as a whole joint project while an agency itself has been making an achievement，further consideration about an appropriate evaluation method is required．
○ As far as self-evaluation is concerned，the range of self-evaluation should be expanded to issues and their responses relating to an agency’s R&D as well as a result of self-evaluation．The reasons for a rating should be understandable and explicit．In addition，further ingenuity to deepen discussions is required．
○ In order to promote seamless R&D，it is desired that researchers should be motivated to match their objectives and research orientation to a given agency．
○ The ideal way to attain “maximization of research and development achievements” should be explored considering differences of the type of researches，and the idea obtained should be reflected on design or evaluation of medium to long-term objectives／plans．
○ It is necessary for a national research and development agency to take international viewpoints into setting or evaluation of objectives appropriately，aiming at “maximization of research and development achievements．”
○ The National Research and Development Agency conducts R&D by utilizing society’s needs and university seeds，therefore it is considered that research seamlessly develops from basic to applied．As a result，a wide range of partnership between the National Research and Development Agency，universities，and companies should be proactively promoted．
○ As far as the National Research and Development Agency is concerned，the proportion of fixed expenses for maintenance management is usually high． Considering this，if subsidies for operating expenses were reduced uniformly，the fact that this may affect R&D projects should be taken into account．
○ Since the accounting standard for the National Research and Development Agency is principally project-achievement-oriented，flexible management should be considered based on the traits of the R&D that the agency conducts．
○ As far as the nation’s policy realization and the improvement of science and technology’s standards are concerned，what kinds of roles the National Research and Development Agency takes on should be sufficiently considered．
○ Whether “a management team” headed by such an agency’s chief director functions effectively through the nurture of each other’s skill or not is important． Such a director is required to manage the whole organization through the utilization of their “management team” without relying on his／her own management capability．
○ An anonymous written interview（upward feedback）targeted at the managers at the National Research and Development Agency was conducted．This written interview was about asking them how they evaluated the chief director’s leadership and management competency．As a result of conducting a feedback against the director，a positive response was also received，whose comment was that it was meaningful．
科学技術・学術政策局企画評価課