Professional Documents
Culture Documents
Prepared for:
IEEE
445 Hoes Lane
Piscataway, NJ 08855
Prepared by:
Anthony F. Breitzman, Ph.D.
CHI Research, Inc.
In 2003,the IEEE commissioned CHI Research Inc., a worldwide leader in Intellectual Property consulting,
to take a look at how IEEE-published information influences patents in today’s top technologies.
According to CHI:
• Patents issued by the U.S. Patent office are citing scientific literature, especially IEEE information,
more than ever before
• IEEE-published science was cited more often than that of other scholarly and commercial publishers
• IEEE is the science base for the top patenting organizations
CHI Research’s methodologies have been featured prominently by BusinessWeek magazine, and used
by the U.S. Small Business Association to identify innovative companies.
CHI had already proven that patents which are themselves cited in later patents are a prime indicator of a
company’s growth potential. Says BusinessWeek (March 2004): “Each year since 2000, when CHI began
issuing monthly buy recommendations to institutional investors (subscription price: $15,000 a year), it
has killed the market averages. In 2003, as the average tech-stock mutual fund returned 55.9%, CHI’s
picks returned 162%.”
The complete CHI Research report is reproduced for you in this IEEE white paper.
About CHI Research Inc., the company BusinessWeek called “A Search Engine
For Tech Prospectors”
CHI Research, Inc. is a highly focused research consultancy internationally recognized for its leadership
in the development and analysis of technology and science indicators. Formed in 1968, CHI pioneered
the development of science and technology citation analysis.
Working with the U.S. National Science Foundation, CHI developed the first national-scale bibliometric
indicators of scientific performance in the early 1970s. Since then, CHI has produced all the literature-
to-literature, patent-to-literature and patent-to-patent citation indicators ever used in the congressionally
mandated Science and Engineering Indicators reports published by the U.S. National Science Board.
About IEEE
The IEEE is the world’s largest technical professional society with approximately 360,000 members in
170 countries. Through its members, the IEEE is a leading authority on areas ranging from aerospace,
computers and telecommunications to biomedicine, electric power and consumer electronics. The IEEE
produces 30 percent of the world’s literature in the electrical and electronics engineering and computer
science fields, and has developed more than 900 active industry standards. The organization also sponsors
or cosponsors more than 300 international technical conferences each year. Additional information is
available at www.ieee.org.
The IEEE engaged CHI to measure the impact of IEEE publications on patents that build upon
IEEE and competitor journal science. The key findings are:
1. A substantial amount of patented technology is built upon the foundation provided by scientific
literature. The IEEE provides much of that foundation for patents in the information technology
industry. IEEE publications have provided a significant scientific foundation for technological
development for years, but the IEEE is becoming increasingly important. In the five years
ending in 1997 and the five years ending in 2002 the numbers jumped to 31,228 and 57,357
respectively. That is, the number of references to IEEE papers in the last five years was 267%
higher than it was a decade earlier. Between the same periods, the number of US patents
increased only 76%, indicating that new technology is increasingly dependent upon papers
published in IEEE journals.
2. The increase in referencing from patented technology to IEEE publications is not driven by a few
key journals, but is broad-based. In fact, of the 62 IEEE journal titles covered by this study, all
but four have shown an increase in patent references in the last five and ten year periods.
Moreover, 55% of the IEEE journals have seen their numbers of patent references more than
double.
3. Determining the value of a large set of patents is an expensive and difficult problem. It has been
shown that citation analysis is a reasonable proxy, because high citation has been shown to be
associated with increases in sales, profits, and stock price. We have found that patents that
reference the IEEE are cited 15% more often than patents that do not reference IEEE, after
correcting for differences in age and technology. This 15% difference is statistically significant,
with the chance that the citation difference is due to random events shown to be less than one in
one million. This suggests that it is likely that the technology that builds upon IEEE science is
likely to be of more value than technology than peer technology not built on IEEE science.
4. In general, patents that build on IEEE science are cited more often than patents that do not.
However, for patents that build on a select set of IEEE journals such as T-IP, M-Net, M-CS, T-
SCM, and others the advantage is much greater. In particular, patents that reference T-IP are
cited 53% more than peer patents from the same technology and publication date. Similarly,
patents that reference M-Net and M-CS are cited 45% more than their peers.
5. A comparison of specific IEEE journals and competitor journals was undertaken as well.
Because of the large number of journals compared, the result is difficult to summarize briefly
(see text and Appendix B). However, in general, most of the IEEE journals are referenced in
patents more often than are the competitor journals.
ii
7. Patents of the 18 selected organizations reference IEEE publications more than 37,000 times. In
other words, about 38% of the science references of these organizations are to IEEE publications.
The second place publisher, the American Institute of Physics, is referenced only 12,048 times
or 12.4% of the time.
8. Patents of Intel reference IEEE publications ten times as often as any other publisher.
Specifically, the 1983-2002 Intel patents referenced IEEE 1232 times (52%), and referenced the
second place ACM only 117 times (5%). Each of the other identified publishers was referenced
107 times or fewer from Intel patents.
iii
Section Page
I. Introduction 1
III. Results 3
V. References 10
iv
Figure Page
CHI Research was engaged in this study by the IEEE to measure the impact of IEEE publications on
patented technology. Several questions were addressed, such as: Are the patents that build upon
IEEE science better than those that do not? Are information technology patents more dependent on
IEEE science than on competitor science, and to what extent do key players build upon IEEE science
versus competitor journal science?
Unfortunately, there is not a direct relationship that would indicate that patents that build upon IEEE
science are worth X dollars more than those that do not. This would be ideal, but it is impossible,
since there is no way to quantify how much a patent citation is worth. However, it has been shown
that companies with highly cited patents in general perform better in terms of sales, profits, stock
price, and patent licensing, than companies with weakly cited patents. Thus by showing that IEEE
science has a positive influence on patent citations, it can be inferred that IEEE science indirectly has
a positive influence on these financial indicators.
The evidence that patent citations are positively correlated with financial performance has been
accumulating in the patent literature for some 40 years. A review of key validation studies for
patents citation analysis is contained in Appendix A.
A. Patents that build upon IEEE publications were identified. Trends were examined to see
if recent patents are more likely to build upon IEEE publications than older patents.
B. A citation analysis was undertaken to determine whether patents that built upon IEEE
published science were more highly cited than peer patents.
The first task involved identifying all US patents with at least one front-page reference to an
IEEE publication. This is a difficult task, because there are no real rules or standard formats for
the way an applicant must cite a reference. This results in two major issues.
1. The journal name may appear anywhere in the reference, first, last, after the author,
before the author, etc.
2. The journal name may be abbreviated in dozens of different ways. For example Table 1
below shows some of the ways that the IEEE Journal of Solid-State Circuits shows up in
patent references.
As part of its processing, CHI standardizes each patent reference, so that papers and journals
can be easily linked to patents. Therefore, identifying patents that build upon IEEE science is
easy using our internal databases.
Figure 1 shows that patented technology is increasingly referencing IEEE publications. In the five-
year period ending 1992, there were 15,625 references to IEEE papers from US patents. In the five
years ending in 1997 and the five years ending with 2002 the numbers jumped to 31,228 and 57,357
respectively. That is, the number of references to IEEE papers in the last five years was 267% higher
than it was a decade earlier. During the same periods, the number of US patents increased only 76%,
indicating that new patents are increasingly building upon papers published in IEEE journals.
Figure 2 examines the referencing of individual IEEE journals. Again, the general trend is positive,
with most journals receiving many more references from patents in the most recent period than from
prior periods. Several journals that were not referenced in patents at all a decade ago are now cited
by hundreds of patents, including J-STQE, T-NET, T-SA, and others. Also, many journals that were
cited hundreds of times by patents a decade ago are cited thousands of times by recent patents.
Examples include T-PAMI, L-PT, and Spectrum among others.
All but four of the 62 journals have received more references from patents in the last five years than
from patents from the previous five years, and 34 of the 62 have seen their references increase by
more than 100%. In other words 55% of the IEEE journals are cited more than twice as often in
recent patents, than they were by patents from five years ago (see Figure 3). This again suggests that
the IEEE is becoming increasingly important to current technological developments.
Protecting technology products by trade secret is difficult, because these types of products can be
reverse-engineered. This is why high tech companies like Canon, IBM, Microsoft, and Intel have
more than 100,000 US patents combined. Moreover, for companies with fast moving technologies,
it is crucial to file patents early, so that is why patented technologies will often precede viable
products even in the information technology (IT) industry.
The net result is that the US has a vibrant and growing patent system that issues nearly 200,000
patents per year to inventors from 50+ countries. With such a vast system, it is inevitable that most
of the patents are worth very little, and that only a small number represent truly valuable innovations.
The problem of identifying the small percentage of truly valuable patents has long been recognized.
One of the most useful methods of identifying key patents is by identifying patents that are cited
frequently by other patents. High citation rates have been associated with seemingly unrelated
benefits such as increases in sales, profits, stock prices, licensing revenue, and inventor awards
among other things. A fairly complete literature review of patent citation validation studies can be
found in Appendix A.
Given this, the obvious question then is, are patents that build upon IEEE science more highly cited
than those that do not? If so, we would have a reason to believe that patents that build upon IEEE
science are more valuable than average patents, although we would not be able to quantify exactly
how much more valuable the IEEE linked patents are.
The main result is that patents that build upon IEEE publications are in general cited 15% more often
than peer patents from the same technology and published in the same time period. This overall
result is strong, but performance for some specific journals is even stronger. In order to discuss the
findings more fully, we need to step back a bit and discuss the method used for comparing the
citation impact.
Citations to patents vary by age and technology, so one must be careful when doing citation analysis.
For example a patent in a mature technology such as metal fabrication will receive fewer citations in
general than a patent in a dynamic technology such as consumer electronics. Similarly, a patent that
is five years old will have had more time to accumulate citations than a patent that is only two years
old.
To account for these issues, we took each patent that referenced any of the 62 IEEE journals and
randomly selected a cohort patent from the same technology class and publication year, so that we
had a randomly sampled peer patent for each IEEE linked patent. We obtained statistically
significant results for 42 of the 62 journals using a Wilcoxon signed rank test1. That is, for 42 IEEE
journals there is a significant difference between the citation patterns of the patents that reference
IEEE versus the peer patents that cannot be explained by a random event. The other 20 journals
either had two few patent references to get a statistically significant result, or there is no significant
difference between the citation patterns of the patents that reference the IEEE journals versus the
peer patents.
Note that getting a statistically significant result could mean either the IEEE linked patents are more
cited than the peer patents or less cited. In this case each of the patent sets linked to the 42 IEEE
journals was cited more frequently than its non-IEEE counterpart. Figure 4 shows that the patent sets
of the 42 journals were cited an average of 18% more often than the peer patents, with some journals
1
The Wilcoxon signed rank test is a significance test that can be applied to non-normally distributed large sets. In
this case it is used to determine whether one citation distribution is significantly different from a second. If the null
hypothesis is true, the sum of the ranks of the positive differences should be about the same as the sum of the ranks
of the negative differences, indicating that any difference in the citation impact could be random. See
http://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/signrank.htm for a formal definition.
As an example, consider IEEE Transactions on Image Processing shown at the top of the table.
There are 329 patents from 1983-2002 that reference this journal. These 329 patents are cited 1333
times in subsequent patents, while a random set of 329 patents selected from the same technology
and publication year were cited only 870 times. In this case the IEEE linked patents received 53%
more citations than the peer set. The significance value in this case is less than 0.0001. This
suggests that there is less than a 1 in 10,000 chance that no difference exists between patent that
reference the IEEE journals and those that do not.
Although the remaining 20 IEEE journals provided results that are not statistically significant, for
what it is worth the patents that reference these 20 journals are also slightly higher cited than their
peer group. Also, the patents linked to all 62 journals are cited 15% higher than their peer patents,
and this is a statistically significant result. In fact, the probability that the 15% advantage happened
as a random event is less than one in one million as Figure 4 indicates.
To review, there is much evidence that shows that high citation rates are associated with
increases in sales, profits, and stock prices. Patents that reference IEEE papers tend to be higher
cited than peer patents. It is therefore reasonable to assume that patents that build on IEEE
science, are in general more valuable than patents that do not.
In the previous section we discussed citation analysis by means of comparing a set of patents (those
that reference IEEE) with a random set of peer patents from the same technology class and era. This
allowed us to determine whether the IEEE linked patent set was cited more often than a random set
in a statistically significant way.
In this section, we compare patents that reference IEEE with patents that reference competitor
journals. This is a more difficult problem, because the comparison patents may be in different
technologies than the IEEE counterparts, or as a group they may be older or newer than the IEEE
linked patents, all of which will affect the citation counts.
To correct for these issues we use a citation index that allows for comparing patent sets of differing
ages and technologies in a precise and fair manner. To understand the citation index we need to
define the expected citation rate.
Consider an arbitrary patent in technology X and published in year Y. This patent may have 0, 1, 2,
or 100 or more citations; suppose it has C citations. Now consider all patents published in year Y
and classified in technology X. Suppose there are M such patents and combined they have N
citations. The expected citation rate for any of the patents in year Y and technology X is simply
N/M, and the citation index for our starting patent is just C/(N/M).
The citation index is strongest as an indicator of a set of patents rather than as an indicator of a
single patent. The citation index of a set of patents is just the sum of the citations to the set of
patents divided by the sum of the expected citations for the same set of patents.
Note that as described above, the citation index is normalized to remove age and technology biases
so that it has an expected value of 1.0. Thus a citation index of 1.5 indicates the patents are cited
50% more than expected, a citation index of 0.5 indicates the patents are cited 50% less than
expected and so on. In this way, two or more patent sets of different sizes, technologies, and ages,
can be compared in a reasonable way.
Appendix B shows the total citations, expected citations, and citation index for the top 30 IEEE
journals referenced in patents and for each competitor journal identified by IEEE staff. This
appendix contains a lot of information, so to avoid repetition only some highlights will be discussed
here in order to illustrate how the data can be interpreted.
The first journal shown in Appendix B is IEEE Transactions on Antennas and Propagation, or T-
AP. What is most striking here is not so much the citation index, but that IEEE T-AP is basically the
standard journal in this area. 774 patents reference T-AP, but none of the competing journals are
referenced by more than 86 patents. The total number of patent references to T-AP (5,640) is more
than four times the combined total of all of the competitor journals. In terms of citations we see that
patents citing T-AP are cited 20% (citation index = 1.2) more than expected. This is slightly lower
than the 1.3 for Radio Science, but that is based on only 86 patents. (Note that as patent sets get
larger they tend to regress to the expected value of 1.0).
Also note that the majority of patents that reference Radio Science also reference one or more IEEE
journals. We see for example in Appendix C, that of the 86 patents that reference Radio Science,
57% of these patents also reference at least one IEEE paper.
T-BME is dominant in its area as well. It is referenced more than twice as often as the leading rival,
and the citation index of T-BME patents is 30% above expected, and above the indices of most of
the rivals. Critical Reviews/Begal House is the only journal with a higher patent citation index (1.7),
but this is based on only 51 patents. Moreover 41% of these 51 patents also reference one or
more IEEE simultaneously, so it is difficult to argue that the citation strength of these patents is due
to their connection to Critical Reviews.
IEICE Tran. Elec. has about half as many patents reference it as T-CAS, but the citation index of
these patents is quite high at 1.7. Interestingly, the 315 patents that reference both IEICE Tran. Elec.
one or more IEEE journals have a citation index over 2. This suggests that patents that build upon
both IEICE and IEEE are of higher impact than patents that reference only one or the other.
The results for the rest of the journals in Appendix B are similar. In most cases, except where the
competitor journal is a general journal such as the Journal of Applied Physics, or Applied Physics
Letters, the IEEE publications enjoy a large lead in patent references over the competitor journals. In
addition to the cases mentioned above, T-SMC, T-MIT, T-IT, M-MICRO,
M-CG&A, T-C, T-CE, and M-COMM all enjoy a two to seven fold lead in patent references over the
nearest competitor journal. The interested reader is urged to examine Appendix B in detail for the
journals of interest to him or her.
In the previous two sections, analyses were carried out that examine the citation impact of patents
that reference the IEEE and competitor journals. By looking at who owns most of the patents that
cite to the IEEE and competitors, we see that not surprisingly the biggest users of IEEE and
competitor science are companies in the Information Technology (IT) and Semiconductor industries,
including telecom companies like Lucent, computer companies like IBM, and semiconductor
companies like Intel. Universities with strong IT components like MIT and the University of
California have patents that also frequently reference IEEE and competitor journals.
Of course in the earlier analyses the patent sets were selected because the patents either referenced
IEEE or a competitor journal; the question that remains is, if we select all patents of Intel, or Lucent,
or IBM for example, how often do these companies’ patents reference the IEEE journals versus those
of others.
Appendix D shows the top referenced journals of each of the top IT and Semiconductor companies
and universities. (Note that the university patents are restricted to IT related technology classes,
because both universities have a large number of chemical and biotechnology related patents that
skew the results.) In each case the IEEE journals are quite well represented.
The results of Appendix D are certainly impressive, but they lead to the question of how dependent
are companies like AMD upon publications published by IEEE versus say, Elsevier or Wiley? These
questions are difficult to address unless the publishers of each journal are known. This is not a
problem for IEEE journals since IEEE most often appears in the journal name, but the same is not
true for journals published by Wiley, Elsevier, and others.
For the 8,713 journals covered by the Thomson/ISI database, the publishers can be obtained from the
following web site: http://www.isinet.com/cgi-bin/jrnlst/jlresults.cgi . But there are still several
hundred prominent journals that are not covered by Thomson/ISI. To identify the publishers for
these, we researched any journal that was referenced by 100 or more times total from the top 18
companies and universities. This allowed us to compile the list of top referenced publishers found in
Appendix E.
The first company in Appendix E is AMD. We see that AMD patents reference IEEE 1084 times or
nearly 42% of the time. CMP is the next most referenced publisher with only 201 references (7.8%)
followed by unknown with 141 references (5.1%). Recall again that the unknown component
consists of journals that are referenced fewer than 100 times total by the 18 selected organizations
and are also not part of the 8,713 core journals covered by Thomson/ISI. Note also, that even if all
of the unknown journals were published by CMP, which is extremely unlikely, IEEE would still be
referenced 3 times as often.
Results for the other 17 companies and universities are similar to those of AMD except for Canon.
Canon references the American Institute of Physics more frequently than IEEE, but this may be due
to Canon’s diverse business that included cameras, copiers, consumer electronics, etc. If the Canon
patents were restricted to IT and Semiconductor related patents, the IEEE may well be first.
Appendix E is rather interesting, but is unfortunately a bit long. The key result is therefore
summarized in Figures 5 and 6. Figure 5 shows the number of references to the top 18 publishers
from the 18 selected companies and universities. To eliminate unknown publishers, only journals
that are referenced at least 100 times total by the organizations are included. We see that the IEEE is
referenced 37041 times by the 18 organizations, or three times as often as second place American
Institute of Physics. The same graph is shown as a set of percentages in Figure 6. Note that the total
does not add up to 100%, since there are some 40+ more publishers not shown that each receive
fewer than 1% of the references.
In this report it was shown that the IEEE has a strong and positive impact on patented technology. In
particular patents that build on IEEE science are likely to be of higher quality than peer patents that
do not build on IEEE science. Moreover, IEEE is the biggest and most important contributor of
science to patents from the IT and semiconductor industries. These are the key results. See the
executive summary for a review of all of the main results.
In terms of recommendations, not many can be made. The report shows that the IEEE is doing a
very good job of providing the science base for the IT community.
There are a few analyses that may be done to strengthen this report. All were beyond the scope
of this project, but may be appropriate now. Some examples include:
1. A drill down analysis by technology. We selected 18 top companies based on the frequency
of referencing to IEEE and competitor journals. Most of these companies were IT and
Semiconductor companies, but they also patent in other technologies. Moreover, the
semiconductor patents of other companies were not included. A more thorough analysis that
examines all patents from all companies and universities in technology subcategories such as
semiconductors, telecommunications, computer hardware, software, and optics, might be of
interest. In this way, we could determine where the IEEE truly has core competencies and
where it leads other publishers by the widest margin.
2. A case study that builds on the current results. We know from Appendix E that Intel patents
that reference scientific papers reference IEEE publications 52% of the time. This is a
wonderful result, but it stops there. Questions such as, what are the patented inventions that
build upon IEEE, and what are the specific IEEE publications that provide the science base
are still unanswered. A case study that drills down to the specific patents that use IEEE and
connects those patents to products might be of real interest.
3. Additional research of the unknown publishers in Appendix E. Recall that any journal
referenced at least 100 times by the patents of the 18 selected companies was looked up to
identify the publisher. The 100 cut-off was necessary, because of the labor involved in
identifying the publisher of an arbitrary journal. This still allowed for some 5-10% of
references to have unknown publishers, which unfortunately weakens the result a bit. A
follow-on project that identifies the rest of the unknown publishers would clean this report a
bit and strengthen the already strong results.
10
[1] Albert M., Avery D., McAllister P, and Narin F. Direct Validation of Citation Counts as
Indicators of Industrially Important Patents, Research Policy 20 (1991) 251-259.
[2] Breitzman A, and Narin F. A Case for Patent Citation Analysis in Litigation, The Law
Works, 3 (3) (1996), 10-11 and 26-27.
[3] Breitzman, A., Thomas, P., and Cheney, P., Technological Powerhouse or Diluted
Competence, (Forthcoming in R&D Management).
[4] Breitzman A, and Narin F, US 6175824: Method and apparatus for choosing a stock
portfolio, based on patent indicators. (Patent Application US1999/353613, July 14, 1999).
[5] Carpenter M, Narin F, and Woolf P. Citation Rates to Technologically Important Patents,
World Patent Information 4 (1981) 160-163.
[6] Deng Z, Lev B, and Narin F., Science & Technology as Predictors of Stock Performance.
Financial Analysts Journal, 55 (5), (1999), 20-32.
[8] Hall, B.H., Jaffe, A.B., and Trajtenberg, M. Market Value and Patent Citations: A First
Look. (NBER Working Paper No. W7741. National Bureau of Economic Research, Cambridge,
MA: June 2000).
[9] Harhoff D, Narin F, Scherer F.M. and Vopel K, Citation Frequency and the Value of
Patented Inventions. (Submitted to Reviews Economics & Statistics, March 5, 1998).
[10] Jaffe, A.B., Trajtenberg, Manuel, and Fogarty, Michael S. The Meaning of Patent
Citations: Report on the NBER/Case-Western Reserve Survey of Patentees. (NBER Working
Paper No. W7631. National Bureau of Economic Research, Cambridge, MA: April 2000).
[12] Lanjouw, J.O., and Schankerman, Mark. Stylized Facts of Patent Litigation: Value,
Scope and Ownership. (NBER Working Paper No. W6297. National Bureau of Economic
Research, Cambridge, MA: December 1997).
[13] Lev B., Intangibles Management, Measurement, and Reporting, (Brookings Institution Press,
Washington D.C., 2001).
11
[15] Mogee, M.E., Patent Analysis Methods in Support of Licensing, Proceedings of the Annual
Meeting of the Technology Transfer Society, July 21-23, 1997, Denver, CO.
[16] Mogee, M.E., Patents And Technology Intelligence, in W.B. Ashton and R.A. Klavans
(eds.) Technical Intelligence for Business: Keeping Abreast of Science & Technology,
(Columbus, OH: Battelle Press, 1997).
[17] Mogee, M.E. and Kolar R.G., Patent Co-Citation Analysis Of Eli Lilly & Co. Patents,
Expert Opinion on Therapeutic Patents 9(3) (1999) 291-305.
[18] Narin F. The Strategic Applications of Technology Indicators Based on Patent Citation
Analysis, Patent World, April 1993.
[19] Narin F., Albert M., and Smith V., What Patents Tell You About Your Competition,
CHEMTECH, (February 1993) 52-59.
[20] Narin F. and Breitzman A., Inventive Productivity, Research Policy, 24, (1995) 507-519.
[21] Narin F., Tech-Line Background Paper, in Joe Tidd (ed.) From Knowledge Management to
Strategic Competence, Series on Technology Management (Imperial College Press, 2000).
[22] Oppenheim, C. Do Patent Citations Count? in Blaise Cronin and Helen Barsky Atkins,
(eds.) The Web of Knowledge: A Festschrift in Honor of Eugene Garfield, (AIS Monograph
Series, Information Today, Inc., Medford, NJ, 2000).
[23] Patent and Trademark Office. U.S. Department of Commerce, Technology Assessment
and Forecast. Sixth Report, June 1976.
[24] Patent and Trademark Office. U.S. Department of Commerce, Manual of Patent
Examining Procedures. Section 904.02, 6th edition (1995).
[25] Pavitt, K, Patent Statistics as Indicators of Innovative Activity: Possibilities and Problems,
Scientometrics 7 (1985) 6-17.
[26] Reisner P., A Machine Stored Citation Index to Patent Literature Experimentation and
Planning, in: H.P. Lunh (ed.) Proceedings of Automation and Scientific Communications Annual
Meeting 1963 (American Documentation Institute, Washington, DC, 1965).
12
[28] Thomas, P. A Relationship between Research Indicators and Financial Performance. 6th
International Conference on Science & Technology Indicators, Leiden, The Netherlands, May 24-
27, 2000.
[29] Trajtenberg. M. A Penny for your Quotes: Patent Citations and the Value of
Innovations, Rand Journal, of Economics 21 (1990) 11.
[30] Worcester Polytechnic Institute, Analysis of Highly Cited Patents: Are They
Important? Report prepared for the U.S. Patent Office, (December 1988).
13
70000
60000 57357
# Patent Refs to IEEE Papers
50000
40000
31228
30000
20000 15625
10000
0
1988-92 1993-97 1998-02
Patent Period
0 or Negative Growth
IEEE 37041
AMER INST
PHYSICS 12048
IEE 4583
INST PURE APPLIED
PHYSICS 3471
ELECTROCHEMICAL
SOC INC 3033
ASSOC
COMPUTING MACH. 2540
OPTICAL SOC
AMER 2364
A V S AMER INST
PHYSICS 2326
AMER CHEMICAL
SOC 1857
PENTON 886
*Publishers compiled from all journals referenced 100+ times total from 18 organizations.
IEEE 38.1%
IEE 4.7%
INST PURE APPLIED
PHYSICS 3.6%
PERGAMON-ELSEVIER 1.1%
PENTON 0.9%
*Publishers compiled from all journals referenced 100+ times total from 18 organizations.
The main point of this section is that there is compelling evidence that high citation rates
are associated with the importance of the technological discoveries in the patents being
cited. This does not guarantee that every highly cited patent is of importance, or that a
patent that is not highly cited is of little importance. It does argue, however, that a
company with a portfolio of highly cited patents is more likely to be technologically
successful than one whose patents are cited less frequently. In addition, there is emerging
evidence that citation indicators are indicative of, and in some cases predictive of,
company technological, economic, and stock market success.
When the referencing pattern is turned around, and all of the subsequent citations to a
given patent are tabulated, one obtains the fundamental information used in patent
citation analysis, namely, a count of how often a given patent is cited in later patents.
These distributions tend to be very skewed: there are large numbers of patents that are
cited only a few times, and only a small number of patents cited more than ten times. For
example, for US patents issued in 1988, and cited in the next 7 years, half the patents are
cited 2 or fewer times, 75 percent are cited 5 or fewer times, and only one percent of the
A-1
Validation Studies
There is no official standard by which the importance of a patent may be judged except,
perhaps, for the Federal Court’s designation of "pioneering patents" (discussed below).
Therefore, most of the studies of citation frequency and patent importance are based upon
the opinions of knowledgeable scientists or engineers, or correlations with non-patent
measures. Below is a chronological history of validation studies. It is by no means
exhaustive, but it shows that patent citation analysis has roots that go back to at least the
1960’s.
One of the earliest studies that looked at patent citations as a way of finding important
patents was done by Reisner [26] at IBM, who experimented with the use of citation
analysis to find key patents. By tracing the references from one patent to another, Reisner
found 43 of 60 patents she was looking for.
Computerized citation data covering all U.S. patents first became available in 1975. In
the following year, in the Sixth Technology Assessment and Forecast report [23], the
Patent & Trademark Office, tabulated the patents that were most highly cited and
suggested that "the number of times a patent document is cited may be a measure of its
technological significance."
In 1981 CHI, in a study sponsored by the NSF [5], showed that patents associated with
the IR-100 award received twice as many citations as a control set of patents of the same
age and in the same technologies. The IR-100 award, established by the journal Industrial
Research & Development, [11], “honors the 100 most significant new technical
products--and the innovators responsible for them--developed during the year.” Another
citation validation study was carried out by Worcester Polytechnic Institute students and
the U.S. Patent & Trademark Office. The last sentence in the abstract explains the key
result. “The results were found to support the hypothesis that highly cited patents are
important” [30].
In 1990, Manual Trajtenberg [29] found that patent citation counts were correlated with
an independent measure of the value of an innovation, measured as a function of price
and product attributes, called by economists the social gain stemming from the
innovation. He also pointed out the significant limitations of simple patent counts as a
measure of technological strength.
A validation study of patent citation analysis within an industrial context was carried out
by CHI Research in cooperation with Eastman Kodak Laboratories [1]. In that study, a
collection of nearly 100 Kodak patents in their core area of Silver Halide Technology
were divided into sets of 16 each, and the sets given to senior lab staff for evaluation. The
Kodak evaluators were senior intellectual property staff, senior lab management, and
A-2
The results showed that whether a patent is cited one, two, or three times does not seem
to make much difference in the peer ranking. However, patents cited more than five times
were ranked far more highly by the Kodak staff. Of the 15 respondents in the study, eight
gave this group of patents the highest average rating. This is a statistically significant
result given that, using the binomial model, the probability of this happening randomly is
0.0002.
A 1996 study [2] examined citation frequencies of three different categories of special
patents: patents listed in the National Inventor’s Hall of Fame, patents of Historical
Significance in a list prepared by the U.S. Department of Commerce for the USPTO
bicentennial, and patents that had been adjudged as pioneering patents by the Federal
District Court. The term pioneer patent refers to patents that the courts have deemed to
be so groundbreaking that they deserve a wide interpretation of their claims.
The study found that pioneering patents are cited almost seven times as often as expected
(with the majority of citations occurring before their special designation). Hall of Fame
patents are cited more than six times as often as expected, and historically significant
almost 2.5 times as often as expected. Moreover, the results were not driven by a few
highly cited patents within each set; in fact, of all the patents in all three sets, only one
was cited fewer times than expected.
A 1998 study [9] used an interesting technique to examine importance versus citation
frequency. This study was based on patents on which profitability information--that is the
private value of the patents--was obtained. The authors considered only patents for which
all the fees had been paid to keep the patents in force in Germany for the full 18 years of
the patents. They queried the owners of those patents as to the asset value of the patent--
essentially asking, what is the smallest amount they would have been willing to sell this
patent to an independent third party for in 1980? In the German patent system the two
patents in the highest value category were much more highly cited than the others. In the
U.S. patent system the patent citation frequency of the patents with an estimated value of
$20 million or above were substantially more highly cited than the patents with lower
estimated values.
In 1999, Thomas, [27] found a correlation between patent citation counts and positive
renewal decisions. Essentially, he found that patents for which the renewal fees had been
paid were more highly cited then their counterparts that were allowed to lapse.
Also in 1999, Zhen Deng et al. [6] used a number of patent citation indicators to predict
stock performance. Other studies related to using citation indicators for choosing stocks
include Breitzman & Narin [4] which showed that a portfolio of stocks picked based on
A-3
Increasingly, economists have begun to use measures based on patents and their citations
as indicators of technological output. Only a few are mentioned here. Hall, Jaffe, and
Trajtenberg [8] explored the contributions of R&D spending, patents, and citation-
weighted patents to measures of the financial market valuation of the firms that own the
patents for more than 4800 US manufacturing firms. They found that citation-weighted
patent counts were more highly correlated with market value than patent counts
themselves because of the high valuation placed on firms that hold very highly cited
patents. Jaffe, Trajtenberg, and Fogarty [10] surveyed 1993 US patentees regarding the
importance of their inventions, the extent of their communication with other inventors,
and the relationship of both importance and communication to observed patent citations.
They found a significant correlation between the number of citations a patent received
and its importance (both economic and technological) as perceived by the inventor.
Lanjouw and Schankerman [12] investigated the characteristics of litigated patents. They
found that, compared with a random sample of U.S. patents from the same cohorts and
technology areas, patents that were cited more often were considerably more likely to be
involved in litigation.
A-4
Antennas and Propagation, IEEE Trans. on (T-AP) 774 5640 5005.2 1.2
Radio Science (formerly Journal of Radi / Amer Geophys Union 86 750 555.9 1.3
Microwaves, Antennas & Propagation / IEE 75 327 273.0 1.2
Journal of Electromagnetic Waves & Ap / VSP 9 23 29.9 0.8
Electromagnetics(frmrly International Jo / Taylor & Francis 6 116 76.6 1.5
International Journal of RF & Microwave / John Wiley & Sons 5 7 6.2 1.1
Competitors Combined: 181 1223 941.7 1.3
Medical and Biological Engineering & Co/ IEE 876 12126 10952.6 1.1
Annals of Biomedical Engineering / AIP 219 1590 1538.3 1.0
Biomedical Engineering / Kluwer 206 2930 2328.1 1.3
Journal Biomechanical Engineering / ASME 144 1112 887.8 1.3
Biomedical Instrumentation & Technolog / Hanley & Belfus 92 379 342.4 1.1
Critical Reviews in Biomedical Engineeri / Begell House 51 299 171.9 1.7
Medical Engineering and Physics / Elsevier 20 57 56.9 1.0
Competitors Combined: 1608 18493 16278.1 1.1
Circuits and Systems Part I, 2 Merged, IEEE Trans. 1268 8599 8839.0 1.2
on (T-CAS_M)
Applied Physics Letters / Amer Inst Physics 14067 116630 99150.5 1.2
IBM Journal of Research and Developm / IBM 2107 26593 18608.8 1.4
IEICE Transactions on Electronics / Oxford 519 2940 1774.6 1.7
Circuits, Devices and Systems / IEE 63 458 435.1 1.1
Computer Aided Geometric Design ( for / Elsevier 40 306 267.3 1.1
Analog Integrated Circuits & Signal Proc / Kluwer 18 76 56.8 1.3
International Journal of Circuit Theory a / John Wiley & Sons 17 102 70.7 1.4
Circuits, Systems and Signal Processing / Birkhauser 13 72 61.9 1.2
Formal Methods in System Design / Kluwer 2 9 11.0 0.8
Journal of Circuits, Systems and Compu / World Sci Pub 2 0 1.9 0.0
Finite Elements in Analysis and Design / Elsevier 1 8 10.9 0.7
Competitors Combined: 16849 147194 120449.6 1.2
The SMPTE Journal (formerly Journal o / Soc MP TV Eng 1088 12842 9988.5 1.3
JEI: Journal of Electronics Industry (JEI) / Dempa 3 23 31.0 0.7
Competitors Combined: 1091 12865 10019.5 1.3
Journal of Applied Physics / Amer Inst Physics 9902 83568 66592.0 1.3
Solid-State Technology / Pennwell 2265 23269 16230.2 1.4
Solid-State Electronics / Elsevier 1063 9679 9037.2 1.1
Microelectronic Engineering / Elsevier 629 4183 2887.0 1.4
NEC Research & Development / NEC Creative Ltd. (Dai 387 4082 3154.2 1.3
Semiconductor Science and Technology / IOP 206 1119 1021.8 1.1
International Journal of High Speed Elec / World Sci Pub 27 83 78.7 1.1
Advanced Functional Materials was (Adv/ John Wiley & Sons 1 0 0.1 0.0
Competitors Combined: 14480 125983 99001.2 1.3
Applied Physics Letters / Amer Inst Physics 14067 116630 99150.5 1.2
Electronics Letters / IEE 7309 59037 52636.1 1.1
Journal of Electrochemical Society / Electrochem Soc 6461 58344 42600.4 1.4
Solid-State Electronics / Elsevier 1063 9679 9037.2 1.1
Semiconductor Science and Technology / IOP 206 1119 1021.8 1.1
International Journal of High Speed Elec / World Sci Pub 27 83 78.7 1.1
Compel / MCB Univ Press 11 62 69.6 0.9
Competitors Combined: 29144 244954 204594.3 1.2
Applied Physics Letters / Amer Inst Physics 14067 116630 99150.5 1.2
Electronics Letters / IEE 7309 59037 52636.1 1.1
Japanese Journal of Applied Physics / Institute of Pure & App 6746 50742 44163.4 1.1
Optics Letters / Optical Soc Am 3288 26337 19821.0 1.3
Optics Communications / Elsevier 1367 10271 9152.5 1.1
Microelectronic Engineering / Elsevier 629 4183 2887.0 1.4
Optical & Quantum Electronics / Kluwer 207 1487 1296.0 1.1
Semiconductor Science and Technology / IOP 206 1119 1021.8 1.1
Fiber & Integrated Optics (formerly Inter / Taylor & Francis 75 912 679.6 1.3
Competitors Combined: 33894 270718 230807.9 1.2
Applied Physics Letters / Amer Inst Physics 14067 116630 99150.5 1.2
Journal of Applied Physics / Amer Inst Physics 9902 83568 66592.0 1.3
Physical Review B / Amer Physical Soc 1366 8905 8027.6 1.1
Competitors Combined: 25335 209103 173770.1 1.2
Physics in Medicine & Biology / Inst of Phys Press 560 3834 3597.7 1.1
Computers in Biology & Medicine / Elsevier 71 864 572.7 1.5
Computerized Medical Images & Graphi / Elsevier 63 341 133.5 2.6
Medical Image Analysis / Elsevier 7 0 0.6 0.0
Competitors Combined: 701 5039 4304.5 1.2
Microwave Theory and Techniques, IEEE Trans. on 1905 14318 14021.0 1.0
(T-MTT)
Journal of Microwave Power & Electrom / IMicroPwr 127 1549 965.5 1.6
International Journal of Infrared & Millim
/ Kluwer 80 370 434.1 0.9
Microwaves, Antennas & Propagation / IEE 75 327 273.0 1.2
International Journal of RF & Microwave / John Wiley & Sons 5 7 6.2 1.1
Microwave Engineering Europe (supple / Miller Freeman 3 51 17.6 2.9
Competitors Combined: 290 2304 1696.5 1.4
Nuclear Instruments and Methods in Phy/ Elsevier 1359 8851 7638.6 1.2
Journal of Nuclear Medicine / Soc Nuclear Med 1249 8133 6303.6 1.3
Medical Physics / American Institute of P 871 7393 6108.5 1.2
Nuclear Technology / American Nuclear Soci 231 909 938.7 1.0
European Journal of Nuclear Medicine & / Springer 181 806 668.7 1.2
Applied Radiation and Isotopes / Elsevier 152 419 520.5 0.8
Nuclear Science and Engineering / American Nuclear Soci 60 430 370.3 1.2
Annals of Nuclear Energy / Elsevier 6 18 38.7 0.5
Competitors Combined: 4109 26959 22587.4 1.2
Pattern Analysis and Machine Intelligence, IEEE 1155 7886 6824.8 1.5
Trans. on (T-PAMI)
Optical Engineering / Int Soc Op Eng 2200 17551 14707.2 1.2
Pattern Recognition Letters / Pergamon/Elsevier 293 3664 2855.7 1.3
International Journal of Computer Vision / Kluwer 178 1187 601.3 2.0
Machine Learning / Kluwer 83 506 359.7 1.4
Image & Vision Computing / Elsevier 56 322 183.5 1.8
International Journal of Pattern Recognit / World Sci Pub 42 265 204.8 1.3
Machine Vision and Applications / Springer 29 350 151.0 2.3
Graphical Models (formerly Graphical M / Academic Press 0 0 0.0 0.0
Competitors Combined: 2881 23845 19063.2 1.3
Japanese Journal of Applied Physics / Institute of Pure & App 6746 50742 44163.4 1.1
Journal of the Optical Society of Americ / Optical Soc Am 2517 22226 16148.0 1.4
Optoelectronics / IEE 44 200 163.8 1.2
Applied Physics B: Lasers & Optics / Springer 12 94 51.6 1.8
Progress in Quantum Electronics / Elsevier 12 76 62.4 1.2
Quantum Electronics / Turpion 0 0 0.0 0.0
Competitors Combined: 9331 73338 60589.2 1.2
Signal Processing, IEEE Trans. on (T-SP) (Includes 2433 23818 21696.7 1.2
Signal Processing Letters at no charge)
Electronics Letters / IEE 7309 59037 52636.1 1.1
Signal Processing / Elsevier 331 2478 1744.8 1.4
Digital Signal Processing (formerly Journ/ Academic Press 83 512 360.3 1.4
Radar, Sonar & Navigation / IEE 22 9 17.1 0.5
Circuits, Systems and Signal Processing / Birkhauser 13 72 61.9 1.2
Sampling Theory in Signal and Imaging / Sampling Publishing 0 0 0.0 0.0
Competitors Combined: 7758 62108 54820.2 1.1
Software: Practice and Experience / John Wiley & Sons 493 5855 3297.3 1.8
Computer Journal / Oxford Univ Press 305 4182 2358.0 1.8
ACM Transactions: Programming Langu / ACM 290 2309 1574.7 1.5
Information Processing Letters / Elsevier 210 3272 2324.6 1.4
Journal of Systems & Software / Elsevier 55 429 403.0 1.1
Acta Informatica / Springer 48 668 482.7 1.4
ACM Transactions: Software Engineerin / ACM 21 70 85.6 0.8
Information & Software Technology / Elsevier 19 137 118.3 1.2
Formal Aspects of Computing / Springer-Verlag 8 33 31.2 1.1
Advances in Engineering Software (form / Elsevier 3 42 31.5 1.3
Automated Software Engineering / Kluwer 2 6 11.2 0.5
Programming and Computer Software / Plenum Press 2 10 3.0 3.3
International Journal on Software Tools f / Springer 1 0 0.1 0.0
Requirements Engineering / Springer 1 12 8.3 1.4
Empirical Software Engineering / Kluwer 0 0 0.0 0.0
Competitors Combined: 1458 17025 10729.7 1.6
Systems, Man, and Cybernetics, Part A,B,C - 692 5883 5052.4 1.4
Merged, IEEE Trans. on (T-SMC_M)
Biological Cybernetics / Springer 154 1532 1585.4 1.0
Management Science / Inst Mgmt Sci 53 348 304.2 1.1
Fuzzy Sets & Systems / Elsevier 49 399 373.3 1.1
International Journal of Intelligent Syste / John Wiley & Sons 41 557 243.1 2.3
Machine Vision and Applications / Springer 29 350 151.0 2.3
Journal of Computer and System Scienc / Academic Press 25 304 278.6 1.1
International Journal of Human-Comput / Elsevier 15 74 24.7 3.0
Journal of Dynamic Systems, Measurem / ASME 11 53 20.9 2.5
Decision Support Systems / Elsevier 5 13 22.5 0.6
Cybernetics & Systems International J/ Taylor & Francis 4 67 10.6 6.3
Control and Cybernetics / Polish Academy of Sci 1 0 1.1 0.0
Information Systems Research / INFORMS 0 0 0.0 0.0
Organization Science / Maruzen 0 0 0.0 0.0
Competitors Combined: 387 3697 3015.4 1.2
# Patent Percent of
Company/University Publisher Refs to total
Canon AMER INST PHYSICS 1130 16.1
Canon IEEE 867 12.3
Canon unknown 804 11.4
Canon ELSEVIER SCIENCE 612 8.7
Canon INST PURE APPLIED PHYSICS 349 5.0
Canon AMER CHEMICAL SOC 232 3.3
Canon ELECTROCHEMICAL SOC INC 226 3.2
Canon ACADEMIC PRESS/ELSEVIER SCIENCE 200 2.8
Canon AMER SOC MICROBIOLOGY 174 2.5
Canon TAYLOR & FRANCIS 179 2.5
Canon IEE 152 2.2
Canon OPTICAL SOC AMER 158 2.2
Canon PERGAMON-ELSEVIER 145 2.1
Canon Russian Academy of Sciences 128 1.8
Canon AMERICAN PHYSICAL SOC 125 1.8
Canon Vacuum Society of Japan 121 1.7
Canon A V S AMER INST PHYSICS 123 1.7
Canon SPIE-INT SOCIETY OPTICAL ENGINEERING 78 1.1
Canon ASSOC COMPUTING MACHINERY 72 1.0
104 others w/ less than 1% 1159 16.5
# Patent Percent of
Company/University Publisher Refs to total
Hitachi IEEE 1980 25.5
Hitachi unknown 935 12.1
Hitachi AMER INST PHYSICS 680 8.8
Hitachi ELSEVIER SCIENCE 454 5.9
Hitachi AMER CHEMICAL SOC 296 3.8
Hitachi INST PURE APPLIED PHYSICS 254 3.3
Hitachi IEE 226 2.9
Hitachi A V S AMER INST PHYSICS 202 2.6
Hitachi ASSOC COMPUTING MACHINERY 184 2.4
Hitachi AMERICAN PHYSICAL SOC 168 2.2
Hitachi ACADEMIC PRESS/ELSEVIER SCIENCE 155 2.0
Hitachi OPTICAL SOC AMER 121 1.6
Hitachi PENNWELL PUBL 110 1.4
Hitachi PERGAMON-ELSEVIER 103 1.3
Hitachi JOHN WILEY & SONS 97 1.3
Hitachi NATURE PUBLISHING GROUP 98 1.3
Hitachi IBM CORP 95 1.2
Hitachi AMER ASSOC ADVANCEMENT SCIENCE 85 1.1
136 others w/ less than 1% 1507 19.4
# Patent Percent of
Company/University Publisher Refs to total
IBM IEEE 4660 29.2
IBM unknown 1906 11.9
IBM AMER INST PHYSICS 1526 9.6
IBM ASSOC COMPUTING MACHINERY 1039 6.5
IBM ELSEVIER SCIENCE 983 6.2
IBM IBM CORP 727 4.6
IBM AMERICAN PHYSICAL SOC 361 2.3
IBM A V S AMER INST PHYSICS 352 2.2
IBM ELECTROCHEMICAL SOC INC 345 2.2
IBM CMP MEDIA 264 1.7
IBM PERGAMON-ELSEVIER 254 1.6
IBM PENNWELL PUBL 224 1.4
IBM JOHN WILEY & SONS 219 1.4
IBM INST PURE APPLIED PHYSICS 231 1.4
IBM IEE 221 1.4
IBM AMER CHEMICAL SOC 225 1.4
IBM ACADEMIC PRESS/ELSEVIER SCIENCE 162 1.0
IBM SPIE-INT SOCIETY OPTICAL ENGINEERING 152 1.0
144 others w/ less than 1% 2109 13.2
# Patent Percent of
Company/University Publisher Refs to total
Lucent IEEE 5317 35.1
Lucent AMER INST PHYSICS 1798 11.9
Lucent IEE 1379 9.1
Lucent unknown 1120 7.4
Lucent OPTICAL SOC AMER 716 4.7
Lucent ELSEVIER SCIENCE 645 4.3
Lucent LUCENT TECHNOLOGIES 596 3.9
Lucent ASSOC COMPUTING MACHINERY 256 1.7
Lucent AMERICAN PHYSICAL SOC 250 1.7
Lucent A V S AMER INST PHYSICS 253 1.7
Lucent INST PURE APPLIED PHYSICS 241 1.6
Lucent ELECTROCHEMICAL SOC INC 224 1.5
Lucent AT&T 210 1.4
Lucent PERGAMON-ELSEVIER 179 1.2
133 others w/ less than 1% 1963 13.0
# Patent Percent of
Company/University Publisher Refs to total
Micron Technology IEEE 3381 36.9
Micron Technology AMER INST PHYSICS 1043 11.4
Micron Technology ELSEVIER SCIENCE 745 8.1
Micron Technology ELECTROCHEMICAL SOC INC 601 6.6
Micron Technology unknown 504 5.5
Micron Technology INST PURE APPLIED PHYSICS 487 5.3
Micron Technology MRS Materials Research Society 263 2.9
Micron Technology A V S AMER INST PHYSICS 250 2.7
Micron Technology SPIE-INT SOCIETY OPTICAL ENGINEERING 221 2.4
Micron Technology IECE 220 2.4
Micron Technology IBM CORP 134 1.5
Micron Technology AMER CHEMICAL SOC 140 1.5
Micron Technology PENNWELL PUBL 99 1.1
Micron Technology PERGAMON-ELSEVIER 98 1.1
Micron Technology AM VACUUM SOC, AM INST PHYSICS 91 1.0
Micron Technology REED BUSINESS INFORMATION/ELSEVIER 93 1.0
61 others w/ less than 1% 790 8.6
# Patent Percent of
Company/University Publisher Refs to total
Mitsubishi Electric IEEE 2118 40.8
Mitsubishi Electric AMER INST PHYSICS 620 11.9
Mitsubishi Electric unknown 421 8.1
Mitsubishi Electric ELSEVIER SCIENCE 261 5.0
Mitsubishi Electric INST PURE APPLIED PHYSICS 250 4.8
Mitsubishi Electric IEE 245 4.7
Mitsubishi Electric ELECTROCHEMICAL SOC INC 132 2.5
Mitsubishi Electric A V S AMER INST PHYSICS 96 1.8
Mitsubishi Electric PENNWELL PUBL 85 1.6
Mitsubishi Electric PERGAMON-ELSEVIER 72 1.4
Mitsubishi Electric IBM CORP 59 1.1
Mitsubishi Electric OPTICAL SOC AMER 52 1.0
Mitsubishi Electric SPIE-INT SOCIETY OPTICAL ENGINEERING 54 1.0
88 others w/ less than 1% 726 14.0
# Patent Percent of
Company/University Publisher Refs to total
NEC IEEE 2317 37.5
NEC AMER INST PHYSICS 712 11.5
NEC IEE 390 6.3
NEC unknown 383 6.2
NEC INST PURE APPLIED PHYSICS 272 4.4
NEC ELSEVIER SCIENCE 259 4.2
NEC IECE 149 2.4
NEC OPTICAL SOC AMER 140 2.3
NEC AMERICAN PHYSICAL SOC 123 2.0
NEC A V S AMER INST PHYSICS 114 1.8
NEC ELECTROCHEMICAL SOC INC 103 1.7
NEC NATURE PUBLISHING GROUP 84 1.4
NEC ASSOC COMPUTING MACHINERY 79 1.3
NEC PERGAMON-ELSEVIER 83 1.3
NEC AMER CHEMICAL SOC 66 1.1
NEC SPIE-INT SOCIETY OPTICAL ENGINEERING 62 1.0
85 others w/ less than 1% 836 13.5
# Patent Percent of
Company/University Publisher Refs to total
Texas Instruments IEEE 2882 41.6
Texas Instruments AMER INST PHYSICS 706 10.2
Texas Instruments unknown 508 7.3
Texas Instruments ELSEVIER SCIENCE 311 4.5
Texas Instruments ELECTROCHEMICAL SOC INC 307 4.4
Texas Instruments A V S AMER INST PHYSICS 269 3.9
Texas Instruments IEE 217 3.1
Texas Instruments PENNWELL PUBL 185 2.7
Texas Instruments MRS Materials Research Society 159 2.3
Texas Instruments AM VACUUM SOC, AM INST PHYSICS 151 2.2
Texas Instruments PENTON 148 2.1
Texas Instruments INST PURE APPLIED PHYSICS 116 1.7
Texas Instruments OPTICAL SOC AMER 113 1.6
Texas Instruments PERGAMON-ELSEVIER 89 1.3
Texas Instruments SPIE-INT SOCIETY OPTICAL ENGINEERING 82 1.2
Texas Instruments TAYLOR & FRANCIS 78 1.1
72 others w/ less than 1% 604 8.7
# Patent Percent of
Company/University Publisher Refs to total
U California Information Technology IEEE 341 19.6
U California Information Technology AMER INST PHYSICS 272 15.6
U California Information Technology ELSEVIER SCIENCE 166 9.5
U California Information Technology AMER CHEMICAL SOC 136 7.8
U California Information Technology unknown 102 5.9
U California Information Technology AMERICAN PHYSICAL SOC 89 5.1
U California Information Technology AMER ASSOC ADVANCEMENT SCIENCE 70 4.0
U California Information Technology NATURE PUBLISHING GROUP 50 2.9
U California Information Technology OPTICAL SOC AMER 46 2.6
U California Information Technology IEE 44 2.5
U California Information Technology INST PURE APPLIED PHYSICS 37 2.1
U California Information Technology IOP PUBLISHING LTD 29 1.7
U California Information Technology ACADEMIC PRESS/ELSEVIER SCIENCE 30 1.7
U California Information Technology PERGAMON-ELSEVIER 27 1.6
U California Information Technology MRS Materials Research Society 23 1.3
U California Information Technology ELECTROCHEMICAL SOC INC 21 1.2
U California Information Technology A V S AMER INST PHYSICS 18 1.0
62 others w/ less than 1% 240 13.8
IEEE publishes the top eight journals in electrical and electronics engineering and the top seven journals
in telecommunications, as well as the number-one journals in many other fields, according to results
from the Institute for Scientific Information’s annual Journal Citation Reports. www.ieee.org/citations.
The annual IEEE Journal Pricing study finds that IEEE magazines, transactions, journals and letters
continue to be priced significantly lower than those of similar publishers. Based on a statistically aver-
age 500-page journal, the results show that the average price of an electrical engineering or computer
science publication was US$644, while the average price of an IEEE periodical was US$391. IEEE
outperformed by providing journals priced 39% less than the market average.
A study published in Library Journal finds that IEEE publications are priced well below market value.
The Library Journal report compared scientific periodicals solely on price, finding that the average
price of an engineering publication in 2004 is $1,491, with computer science publications averaging at
$1,171. The average unequalized price of an IEEE periodical in 2004 is just $509.
Users of IEEE technical information downloaded a record 43.7 million full-text PDFs from the IEEE
Xplore delivery system in 2003 – an increase of 88.7% over downloads in 2002.
If your company had instant access to IEEE online publications, how much more could you accomplish?
www.ieee.org/discover.