L do lm bo co ny

nh gi nh lng kt qu nghin cu khoa hc q g

Chia s mt s thng tin ti tm hiu thi gian qua v cu chuyn nh gi kt qu nghin cu khoa hc. Cng trao i mt s suy ngh v vic dng nh g y g g gi nh lng mt cch thch hp trong hon cnh c th ca ta.

H T Bo Trng Khoa hc Tri thc g Vin Khoa hc v Cng ngh Tin tin Nht Bn
2

Ni dung

Ba loi hnh nghin cu

Nghin cu c bn Nghin cu ng dng Nghin cu pht trin

M u V cc o

Ch s trch dn H s h h ca t ch nh hng tp h Ch s H

Tm tri thc khoa hc nn tng v g thin nhin v x hi. Chng minh c on Poincar trong ton hc hay g y vic xc nh cc gien gy ra bnh tt.

Tm tri thc khoa hc mi v cn cho nhu cu thc tin. Cch chn on cp bnh vim g gan da trn xt nghim mu nhng khng lm sinh thit.

Tm tri thc mi lm ra cc sn phm c th (thng ca doanh nghip). Hng Microsoft nghin cu lm h g iu hnh my tnh Windows 7.

Mt vi nhn xt v kin

Sn phm ch y l cc n p p yu phm khoa hc (bi bo, sch chuyn kho, bo co k thut,) 3

Sn phm ch y l tri p yu thc n trong sn phm 4

Kinh ph cho cc loi hnh nghin cu Nht

Nghin N hi cu c bn i hc v cao hc Vin phi li nhun h Vin nghin cu cng Doanh nghip T l chung Nghin N hi cu ng dng Nghin N hi cu pht trin

nh gi n phm khoa hc

nh gi s lng nh gi cht lng n phm

55,1 55 1 20,3 , 24,4 6,3 14,3

35,8 35 8 35,8 , 29,6 19,6 22,8

9,1 91 43,9 , 46,0 74,1 62,9

5

nh gi ch quan (peer review, nh gi nh tnh) nh gi khch quan (metrics-based, nh gi khch quan) Ch s trch dn (citation index) ( ) H s nh hng (impact factor) Ch s H (h-index). ( )
6

o nh lng tiu biu:

o g v cho ai?

C s d liu thng tin khoa hc

Tng s bi bo (total paper) Tng s trch dn (total citations) Trch dn ca tng bi bo (cites per paper) T l bi c trch dn (percent cited paper) nh hng tng i trong ngnh (impact relative to field) Xp hng trong ngnh (percentile rank in field) Ch s hp tc (collaboration indicators) S trch dn trung bnh (expected citation count) T l trch dn trn trch dn trung bnh (ratio of citations to expected citation count) Chi s H (H index) Phn b tn sut trch dn (citation frequency distribution) ) etc.

Tc gi Authors n v Institutions Quc gia Nations Ch Topics Lnh vc Fields

C s d liu ISI (Institute for Scientific Information, Thomson-Reuteur 4.2008) C s d liu SCOPUS (Elsevier) C s d li G liu Google S h l (G l Scholar (Google) l ) Hn 100 c s d liu v cng c khc cho php g p p tm kim ch s trch dn, nh arXiv, CiteSeer, ScienceDirect, SciFinder Scholar, MEDLINE,

C s d liu ISI

Lch s Web of Science

Science Citation Index Arts & H manities Humanities Citation Index
Web of Science:

SCI (Science Citation Index): t 1964, 3773 tp ch ca 100 ngnh; v SCIE (SCI m rng, Science Citation Index Expanded) vi 8207 tp ch ca 150 ngnh. SSCI (Social Sciences Citation Index): t 1973, hin c 2697 tp ch v 3500 cng trnh ca 50 ngnh; A&HCI (Arts & Humanities Citation Inde): t 1978, hin c 1470 tp ch v 6000 cng trnh khc. CPCI (Conference Proceedings Citation Index): 110,000 y p g tuyn tp 256 ngnh khoa hc t nhin, x hi nhn vn Chuyn ln mng (1997) di tn Web of Science ( p // (http://isiwebofknowledge.com). g )
9

v3

v4

v5 v6

v7

1960

1965

1970 Social Sciences Citation Index

1975

1980 SciSearch

1985

1990 SCI-CD

1995

2000

2005 Century of Science

2007

ISI Web of Knowledge

(from David Horky, Thomson-Reuteurs)

10

Journal citation reports

Journal Citation Reports

Scopus and Google Scholar

Journal A Journal B 01100100101011 01001100110110 Journal C 01100100101011 10010010011101 01001100110110 10100100110101 01100100101011 10010010011101 01001100110110 10100100110101 10010010011101 10100100110101 0 00 00 0 0

Web f W b of Science

SCOPUS (http://info.scopus.com, t 2004)

16.500 tp ch, 600 phm ngh nghip, n 350 lot sch chuyn kho, ~3,6 triu bi bo t hi ngh.

Article level data.

Journal level data.

Google Scholar (http://scholar.google.com, t 2005)

Web of Science cha thng tin th mc v trch dn cc bi bo trong cc tp ch, c th c x l to ra nhiu o nh gi cc tp ch nh o mc nh hng xc nh ch, hng, xu th, phn v c trng cc tp ch.
(from Thomson Reuters: Journal Citation Reports and the Impact Factor)

hu ht cc tp ch ca nh xut bn ln u-M, u M, bo co k thut, lun vn, sch v nhiu loi ti liu khc.

11

12

Ch s trch dn

Ch s trch dn
Mt s c im

2004 2006
Times Times Cited Cited

S ln c trch dn, tham kho trong cc n phm khc (Garfield, 1955)

Ch c ngha so snh trong tng ngnh khoa h h h h kh hc. Thng khc nhau v c sai s khi tnh t cc ngun khc nhau h S lng bi bo cng nh s citations ca cc ngnh to nh l rt khc nhau (commutative algebra vs machine to-nh vs. learning). Khng d xc nh v phn bit self citation self-citation. Mt s ngnh nh arts, humanities hay math, citations thng n mun chm nhng trong thi gian di mun, di. ngnh khc nh life sciences, citations thng nhanh, nhanh t nh trong vng 2-3 nm ri thi 23 thi. n
14

The Concept of Citations

2003

2007

Cited References

20 2000 00
Related Related Records Records R d

2000

2004

1974

1998

1993 1999 2002 1994

(Eugene Gafield, Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas, Science, 122, 3159 (July 15, 1955):108)
13

Ch s trch dn

Ch s trch dn

Ch s trch dn ch c ngha so snh trong tng ngnh khoa hc. Ch s trch dn tnh t cc ngun khc nhau thng khc nhau v c sai s.

Khc bit v vn ha ngnh. Ph thuc vo , bn cht, cch lm v cch cng b. Khc nhau gia nghin cu l thuyt thc th t v th nghim.

S trch dn trung bnh trn mt bi bo

Ton hc v tin hc Khoa hc x hi Khoa hc vt liu Sinh hc Khoa hc v mi trng Khoa hc v tri t Ha hc Vt l Dc liu Y hc lm sng Khoa hc v b no Khoa hc v s sng

S trch dn

15

(http://www.elsevier.com/framework_editors/pdfs/Perspectives1.pdf)
16

Ch s trch dn

Ch s trch dn

Ch s trch dn tnh c t cc ngun khc nhau g thng khc nhau v c li.

Precision ( chnh xc) v recall ( ( ) (kh nng tm ht) g ) Li tnh ton thng do tn ngi trng hoc ging nhau.

Sch Quantum Computation and Quantum Information Quantum Information (2000, Cambridge Univ. Press) c trch dn 2800 citations t Web of Science 3150 t Scopus, 4310 t Google (gp 1,53 so vi WoS) 283 bi bo t 3 tp ch y hc c trch dn 68088 t Web of Science 82076 t S Scopus 83538 t Google Scholar 1,226 ln cao hn WoS)

Nguyn Anh Tun c 80 bi bo c trch dn Nguyn Anh Tun (khc) c 5 Nguyn Anh Tun c 35 bi Google tm c 100 bi uc trch dn vi N.A. Tuan, trong s ny c 60 bi ca Nguyn Anh Tun Precision = 60/100 = 0.6 Recall = 60/80= 0.75

retrieved

relevant

precision

retrieved relevant retrieved ti d

recall

retrieved relevant relevant

retrieved {retrieved documents} relevant {relevant documents} { }

(Kulkarni, A.V., et al., Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals, JAMA, 302(10), 2009.) 17

18

H s nh hng ca tp ch (IF)

H s nh hng khc nhau gia cc ngnh

Y hc (1) A Cancer Journal for Clinicians (2)The New England Journal of Medicine (100) Epilepsia 3,733 [739] 74,575 [1] 50,017 [2] Ton hc (1) Communications on Pure and Applied Mathematics (2) Bulletin of the American Mathematical Society (100) Monatshefte Fur Mathematik 0,584 [5248] 3,806 [711] 3,5 [851]

IF ca tp ch T trong nm N tnh bng IF = A/B A/B,

A = s trch dn nm N n bi ca T trong hai nm ngay trc N B = tng s cc bi ng trn T trong hai nm ny. g g g y
s o tri c h d a n tr o n g 2 0 0 9 d e n b a i c u a t a p c h i d a n g tr o n g 2 0 0 7 v a 2 0 0 8 s o b ai c u a t a p c hi d a n g tr o n g n a m 2 0 0 7 v a 2 0 0 8

I F 2 0 0 9

IF 2009 l 7,180: trung bnh mi bi ng nm 2007 v 2008 t h d 7 180 l t c trch dn 7,180 ln trong 2009 IF c th thay i nhiu theo thi gian (Bioinformatics)
2008 4,328 2007 4,894 2004 5,742 2003 6,701 2002 4,615 2001 3,421 19

S trong (x) v [y] l th t ca IF trong chuyn ngnh v ton b cc ngnh.

Th t IF khng lun ng nht vi uy tn ca tp ch trong chuyn ngnh (Annals of Mathematics uy tn nht trong Ton) C th sai g khi so snh th s IF gia cc ngnh?
(Journal Citation Reports ca Web of Science, 2008)

20

H s nh hng trung bnh theo ngnh

Gi s IF tng tuyn tnh trong cc ngnh, th c th ni nm na rng IF 9,5 ngnh sinh h h i h hc phn t v t bo c nh hng ngnh ny qung nh nh hng ca mt tp ch c IF 6 trong g ngnh y, tp ch IF 4 trong vt l hay mt tp ch c IF 1,2 trong ton v tin hc.
0,631
Tin hc

Vi c im ca h s nh hng

0,556 0 556 1,912

Vt l

Gi tr trung bnh v nh hng ca cc bi trong tp ch

Ton hc

2,610

Theo th t trch dn: 15% bi u gp 50% s trch dn, 50% bi u gp 90% s trch dn, cc bi nhm 50% ng u c trch dn khong 10 ln hn cc bi nhm 50% cui.

Ha hc

3,252
Khoa hc v no

Tnh IF trong 2 nm sau cng b l ngn v sm i vi mt s ngnh JCR thm IF tnh trong 5 nm. h h h Cha tnh bi c dng nhng khng c trch dn Bin ng ng k t nm ny qua nm khc Qung 10 15 nghn trn 110 nghn tp ch tham gia vo 10-15 vic tnh h s nh hng. Tp ch khng dng ting Anh v cc ngnh nh t c tnh n IF.
22

4,763 4 763 2,896

Y hc Sinh hc phn t & t bo

H s nh hng trung bnh ca tp ch trong cc ngnh

21

kin chuyn gia

(Philip Campbell, Editor-in-chief of Nature)

Natures IF for 2004 was 32,2 89% of our impact factor was Nature s 32 2 generated by just 25% of our papers. I am concerned by the tendency within academic I administrations to focus on a journals impact factor when judging the worth of scientific contributions by researchers, affecting promotions, recruitment and, in some countries, financial bonuses for each paper. Although the current system may be effective at measuring merit on national and institutional scales, the most effective and fair analysis of a persons contribution derives from a direct person s assessment of individual papers, regardless of where they were published.
23

H-index

Cn bng s lng v cht lng Mt ngi c ch s H l N nu c N bi bo vi ch s trch dn t nht l N, v cc bi cn li c ch s trch dn nhiu nht l N.

Mt ngi cng b 40 bi bo, 9 bi c nhiu hn 10 trch dn, 5 bi c 10 trch dn v 26 bi cn li c t hn 10 trch dn, th ch s H ca ngi ny l 10.

Th

http://delab.csd.auth.gr/~lakritid/index.php?lan=1 &s=2
Hirsch, J.E., An index to quantify an individual's scientific research output, PNAS 102, 2005.

(Escape from the Impact Factor, Ethics in Science & Env. Politics, 8, 5-7, 2008)

24

c im ca ch s H

c im ca ch s H

Ch s H c ngha hn vi nhng ngi thuc cng mt ngnh, khng lun thch hp khi so snh khc ngnh. Khng phn bit c s ng gp khc nhau ca cc tc gi trong mt cng trnh. Bn kiu th t tc gi:

Precision v recall thng khng cao khi tnh t ng do cha phn bit r tn ngi Khng nh gi c cng trnh nh hng rt ln Khng phn bit c l do trch dn mt bi bo:
(a) ( ) (b) (c) (d) (e) () (f) Ghi nhn cng trng ca tc gi h t t i Th hin s knh trng tc gi Ni v phng php lin quan Cung cp thng tin nn c ch Trch dn ph phn Trch dn lm c s cho bi bo

Th t ng gp do cc tc gi xc nh (nh tnh); Th t ch ci ca tn khi cc tc gi xem ng gp mi ngi nh nhau; Th t u-cui, nhn mnh tc gi u v cui l quan trng nht, v nhng ngi gia theo ng gp; Th t ng gp do cc tc gi xc nh mt cch nh lng,

Ch s G: cao nhng bi bo c rt nhiu trch dn

26

Tscharntke et al.Author S Sequence and Credit f Contributions in Multiauthored C for C Publications, PLoS Biology, 5(1), 2007.

Lawani, Citation Analysis and the Quality of Scientific Productivity, BioScience, 27, 1977

25

Mt s nhn xt v trao i
1. Cn hiu r c im v dng ng (hp l) cc o nh gi nh lng
Trnh dng sai khi nh gi cc c th

Mt s nhn xt v trao i
4. Cn ch n hng ca khc bit vn ha ngnh nh ln cc o v s lng n phm
(a) Khc nhau v di (NB Chu 197 trang) (b) Khc nhau v s tc gi khc v s lng (c) Khc nhau v vai tr ca hi ngh khoa hc, th d nhiu hi ngh u ngnh CNTT c nh gi cao nh (hn) nhiu tp ch. C kh t Cn trong h gi hi ngh nh i h ISI Conference Proceedings Citation Index C th (nn) tham kho cc danh sch conference ranking http://core.edu.au/index.php/categories/conference%20rankings
27
28

2. Nn dng kt hp nhiu o nh gi nh lng 3. Cn c thm thm nh ca chuyn gia khi cn

Cha phn bit t trch dn p Trch dn ph phn ng gp ca tc gi trong bi bo

Mt s nhn xt v trao i
5. 5 Nhng sn chi khc nhau v nhng sn chi mi sn chi sn chi
90% bi bo trn tp ch c trch dn = 0, 50% khng bao gi c ai khc c ngoi chnh tc g v nhng ngi p g gi g g phn bin. WASET (World Academy of Science, Engineering and Technology, http://www.waset.org) WSEAS (World Scientific and Engineering Academy and Society, http://www.worldses.org/wseas.htm). Hai kha cnh? (Bit v Cn)
Kumar, M.J., E l ti S i tit Cit ti K M J Evaluating Scientits: Citations, Impact Factor, h-Index, O li Hit I tF t hI d Online Hits and What Else?, IETI Technical Review, 26(3), 165-168, 2009 29

Mt s nhn xt v trao i
6. Khng ngng ci tin cc o
H index G index, s ln download , H s ring (eigenfactor) hc t Pagerank ca Google. Cn lun ch n cc ci tin cc o tt hn tin, hn.

7. Thng tin khoa hc ngy cng phong ph v minh bch

D xc nh o vn, np nhiu ni, bia web vn cn tr tr o c khoa hc y S nhanh nhy ca tui tr
30

Mt s nhn xt v trao i
8. V mt s nh gi tnh hnh nghin cu ca ta

S bi bo c TQT Bn i hc hng u Vit Nam Vin Khoa hc v Cng g ngh Vit Nam Ton b s bi bo ca Vit Nam i hc Chulalongkorn i hc Mahidol 160 156 806 869 817 Tc gi ni a ch tr 87 74 307 715 598 Phn trm ni a 54 47 37 82 73

Mt s nhn xt v trao i
9. V Qu NAFOSTED, SCI v SCIE
C th rt nhiu bi cui SCIE

Cc nghin cu so snh c gi tr tham kho SCImago Journal & Country Rank

http://www.scimagojr.com/inde http://www scimagojr com/inde x.php

10. Lin h vi tnh hnh ca ta

Dng cc o nh lng hp l: hiu v dng chng ng n, v hp vi tnh hnh khoa hc ca ta (ngng) Nng cp mt s tp ch trong nc. Dng thay cho mt phn cho cch tnh im trong xt GS-PGS?

Ngh v nh gi chng hp l?

S cng b quc t nm 2008 ca cc t chc nghin cu v pht trin hng u ca Vit Nam v Thi Lan

Pham Duy Hien, A Comparative Study of Research Capacities of East Asian Countries and Implication for Vietnam, Higher Education, Springer, Feb. 2010.
31

32

Kt lun

Ti liu tham kho chnh

D cn nhng hn ch, tnh c t ng cc o l mt bc tin nhiu ngha v vic s dng chng h ang l xu th t trong pht t i kh h ht trin khoa hc. Hiu r v dng hp l cc o nh lng nh gi kt qu nghin cu c tc dng thc y pht trin khoa hc v cng ngh ca chng ta.

Adler, R., et al. Citation Analysis, Statistical Science, 24(1), 1-14, 2009. Althouse, B.M., et al., Differences in Impact Factor Across Fields and Over Time, American Society for Information Science and Technology, 60(1), 27-34, 2009. Bakkalbasi, N., al Three Bakkalbasi N et al., Three Options for Citation Tracking: Google Scholar Scopus Scholar, and Web of Science, Biomedical Digital Libraries, 3(7), 1-8, 2006. Bergstrom, C.T., Eigenfactor: Measuring the Value and Prestige of Scholarly Journals, Journals College & Research Library News 68(5) 2007 News, 68(5), 2007. H T Bo, T chc v qun l ti nghin cu khoa hc Nht, Tp ch Tia Sng, 8.2008. Campbell, P., Escape from the Impact Factor, Ethics in Science & Environmental Politics, 8, 5-7, 2008. Eugene Gafield Citation Indexes for Science: A New Dimension in Documentation Gafield, Citation through Association of Ideas, Science, 122(3159), 1955. Harzing, A. K, Wal, R., Google Scholar as a New Source for Citation Analysis, Ethics in Science & Environmental Politics Vol 8 61 73 2008 Politics, Vol. 8, 61-73, 2008.

http://www.jaist.ac.jp/ bao/writinginvietnamese http://www jaist ac jp/~bao/writinginvietnamese 33

34

Ti liu tham kho chnh

Hirsch, J.E., An Index to Quantify an Individuals Scientific Research Output, Proceedings of the National Academy of Sciences, 102 (46) 16569 16572 2005 (46), 1656916572, 2005. Iglesias, J.E., Pecharromen C., Scaling the h-index for Different Scientific ISI Fields, Scientometrics, 73(3), 303-320, 2007. Kulkarni, A.V., et al., Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals, The Journal of the American Medical Association, 302(10), 1092-1096, 2009. Kumar, M.J., Evaluating Scientits: Citations, Impact Factor, h-Index, Online Hits and What Else?, IETI Technical Review, 26(3), 165-168, 2009. Lawani S.M. Citation Analysis and the Quality of Scientific Productivity, BioScience, 27, 26-31, 1977. Meho, L.I., The Rise and Rise of Citation Analysis, Physics World, 20(1), 32-36, 2007. Pham Duy Hien, A Comparative Study of Research Capacities of East Asian A Countries and Implication for Vietnam, Higher Education, Springer, Feb. 2010. Tscharntke, T., Hochberg, M.E., Rand, T.A., Resh, V.H., Krauss, J., Author Sequence , gy ( ), and Credit for Contributions in Multiauthored Publications, PLoS Biology, 5(1), 00130014, 2007.

35