An Event Detection Algorithm Based
Li-Qing Qiu Bin-Pang Li-Ping Zhao
State Key Lab. ofSoftware Development Environment, Beihang University, 100083
{qiuliqing, pangbin, zhaolp }, nlsde. buaa. edu. cn
Abstract
In order to overcome some shortcomings of
traditional algorithms in event detection, we propose
an event detection algorithm based on improved STC
(suffix tree clustering), which detects significant
events from large volumes of news and presents the
main content of the events to the user as summaries.
The experimental results on TDT indicate that the
new algorithm is an effective document clustering
algorithm.
1. Introduction
A new event is defined as a specific thing happens
at a specific time and place [ 1], which may
consecutively reported by many news articles in a
period. Automated detection of new events from
Web documents is an open challenge in text mining.
Event detection is an unsupervised learning task.
Document clustering algorithms attempt to group
documents together based on their similarities; the
documents that are relevant to a certain topic will
hopefully be allocated in a single cluster [2].More
specially, the cluster algorithm used in event
detection is incremental. Current event detection
systems are mostly based on comparing a new
document to the clusters of documents in the past,
and thresholding on the similarity scores-if all the
similarity scores are below a threshold, the new
document is predicted as the first story of a novel
event, otherwise, it belongs to the most similar
clusters.
In this paper, we focus on how to use improved
STC to detect new events, which is an incremental, 0
(n) time algorithm that produces coherent clusters.
Our main work includes:
(1)Improving the algorithm of STC
Authorized licensed use limited to: SATHYABAMA UNIVERSITY. Downloaded on November 5, 2008 at 03:37 from IEEE Xplore. Restrictions apply.
on Improved STC
(2)Proposing a new algorithm based on improved
STC for event detection.
The remainder of this paper is organized as
follows. We review the previous research work in
section 2.1n section 3 we summarize STC and
describe the problem of STC. The proposed
algorithm is presented in section 4.Then we report on
experimental methodologies and results in section 5.
At last, we conclude our paper and discuss the future
plans in section 6.
2. Related work
One of the hotspots of event detection is clustering
algorithm. In addition, some researchers proposed
some methods to improve the performance such as
imposing a time window [2] and re-weighting named
entities [3].
Yang et al. [2] proposed a clustering algorithm,
GAC (Group Average Clustering), a divided-and-
conquer version of a group-average clustering
algorithm. GAC performs AHC (Agglomerative
Hierarchical Clustering), producing hierarchically
organized document clustering. AHC does not try to
find "best" clusters, but keeps merging the closest
pair of objects to form clusters. With a reasonable
distance measurement, the best time complexity of a
practical AHC algorithm is O(N2) . So AHC is
typically slow when applied to large Web documents.
Yang et al. [2] and Allan [4] proposed Single-pass
algorithm, which was straightforward. The algorithm
sequentially processes the input documents, once at a
time and grows clusters incrementally. A new
document is absorbed by the most similar cluster in
the past if the similarity between the document and
the cluster is above a preselected clustering
threshold. Single-pass method is very easy, as well as
the most popular clustering algorithm in event
528
 detection, but it suffers from being order dependant Document3: Cat ate mouse too.
and from having a tendency to produce large clusters
[5]. Moreover, it only processes the input documents
once at a time, which is limited in large Web
tc kiNu COU
-.
documents.
Lei et al. [6] proposed an improved incremental K-
means for detecting events. In order to select initial cwc5c Lli
I-1
(II,
im
i. 11)
t'nolis
RIOU-Ne 0
cluster centers objectively, the algorithm utilizes ttx)
\R.W
chr.

\,--l
density function to initialize cluster centers. The u

quantity of clusters is affected little by the order in

which the news stories are processed. But the
initialization of cluster centers adds time and space-
consuming, so it is unreasonable used in on-line
detection.
From the aspect of utilizing the contents of
documents, TF-IDF is still the dominant technique
for document representation. In this method, each
document is represented by a vector of weighted
terms that can be either words or phrases and ignore
the sequence order of the words or phrases[5], thus
losing valuable information.
3. Summarization of STC
STC is proposed by Zamir and Etziomi [5] for
clustering in their meta-search engine, which first
identifies sets of documents that share common
phrases by construction suffix tree, and then creates
clusters according to these phrases. STC does not
treat a document as a set of words but rather as a
string, making use of proximity information between
words. STC relies on a suffix tree to efficiently
identify sets of documents that share common phases
and used this information to create clusters and to
Figurel. An instance of suffix tree
At the same time, suffix tree is trying to keep the
sequential order of each word in the original
documents in order to display the summary in step
(3).The structure can be constructed in time linear
(linear in the size of the document set), and can be
constructed incrementally as the documents are being
read [5].
Each node of the suffix tree represents a group of
documents and a phase that is common to all of
them. The label of the node represents the common
phrase; the set of documents tagging the suffix-nodes
that are descendants of the node represents a base
cluster.
(3) Merging these base clusters into clusters
The final step is merging base clusters with a high
overlap in their document sets, which allows a
document to appear in more than one cluster.
Clusters are scored and a label is generated for each
clusters. Figure 2 is an example of base cluster graph.
Phrase: cat at
Dcument
rlt )1sS 3

succinctly summarize their contents for users. 1)- liraSe i 1nous

D)ocum e
Ph rase: chet

STC has three logical steps as followings:

(1) Document "cleaning"
Sentence boundaries are marked and non-word e.ocmnt'.
token (such as HTML tags) are stripped. The strings Phae too}
of each document are transformed using a stemming P.ira sem ate 4
Ie 3 IDoculerits.
algorithm.
(2)Identifying base clusters using a suffix tree Figure2. The example of base cluster
Suffix tree document model considers a document
to be a set of suffix substrings, the common prefixes Based on above model, we have identified several
of the suffix substrings are selected as phrases to key features of STC, which is excessively suitable for
label the edges of a suffix tree. Figure 1 is an event detection:
example of the suffix tree of a set of documents as (1) Incrementality: The strings associated with
following: each document can be easily inserted onto the suffix
Documentl: Cat ate cheese. tree as soon as the document is received. In contrast,
Document2: Mouse ate cheese too.

529

Authorized licensed use limited to: SATHYABAMA UNIVERSITY. Downloaded on November 5, 2008 at 03:37 from IEEE Xplore. Restrictions apply.
 most cluster algorithms can not process document
sets incrementally including AHC.
(2) Linear time: Unlike AHC, STC is a linear time
clustering algorithm (linear in the size of the
clustering set) which is based on identifying phrases
that are common to groups of documents.
(3) Overlapping clusters: STC creates overlapping
clusters, namely, it allows one document belong to
more than one cluster, which is more reasonable
because one document may have multiple topics.
(4) Non-exhaustive: Unlike Single-pass algorithm
which processes the input documents once a time,
STC can process a batch of documents easily by
inserting the strings of each document onto the suffix
tree.
(5) Browsable summaries: One or more several
phrases are naturally selected to generate a topic
summary to label the corresponding cluster clusters
during building the clusters in STC. However, it can
not provide concise and accurate descriptions of the
clusters in most clustering algorithm.
However, STC has some shortcomings. Chim et al.
[7] proposed that there was no efficient measure to
evaluate the quality of clusters in STC. Branson et al.
[8] proposed that it was unnecessary presenting all
labels to the users, because it was common that labels
that were subsets of one another to be merged into
the same cluster.
In the paper, the measure to evaluate the quality of
clusters in STC will be improved. Furthermore,
labels that will present to the users will be improved.
4. Improvement of STC
We implement a modified version of STC. The
input to our algorithm is a collection of documents
and a set of user-specified parameters. The output is
a forest of trees of clusters. The main steps are
described in section 3. We make following changes
to the algorithm in section 3.
(I)For merging base clusters, STC defines a binary
similarity measure between base clusters based on
the overlap of their document sets. Given two base
clusters Sm and Bn , with sizes
and IBnrespectively, and representing the number
of documents common to both base clusters. The
similarity of Bm and Bn to be 1 if:
BnBn|/ IB > 0.5 and
Authorized licensed use limited to: SATHYABAMA UNIVERSITY. Downloaded on November 5, 2008 at 03:37 from IEEE Xplore. Restrictions apply.
|Bm0B |/|Bnl > 0.5
Otherwise, their similarity is defined to be 0.
We found that the "and" Boolean operator is not
suitable in the following condition if one base cluster
is the subset ofthe other:
Ba cBn andB < 0.5
or
BnaCBm and B| < 0 5 Bm
So we change the "and" operator to "or" operator:
|BmnBn|/ gBm| > 0.5 or
|Bm0B /Bn > 0.5
This is essentially identical to Yang's [10]
improvement, except that we take place the threshold
as 0.5 rather than an undefined parameter.
(2)Cluster score is very important, however, Zamir
et al. [5] does not describe how to score the cluster.
We score the clusters using following function:
Score(B,) = B, /min(Length(Label, )) Weight(doc)
* *
Where IBX is number of documents in cluster
Bx, min(Length(Label1)) is the minimum length of
labels of cluster Bq, Weight(doc) is the weight of the
documents which Bq belong to.
(3) Clusters remain at least half of the nodes
which will contain the main idea of the documents.
Therefore any label or combination of labels in the
merged cluster should be a good general description
of the documents in the cluster. However, it is noted
that some labels are the subsets of one another, which
may be merged into the same clusters. We do not
want to double count them. More specially, we
choose the longest label first, and then choose the
labels which are not the subsets of any selected
labels.
(4) Chim et al. [7] proposed a novel method which maps
all nodes n of the common suffix tree to a
M dimensional space of VSD (Vector Space Model)
(n = 1,2, , M ),each document d can be represented
as a feature vector of the weights of M nodes:
d = {w(l, d), w(2, d), w(M, d)} .,
IBm The document frequency of each node df(n) is
defined as the number of the different documents that
have traversed node n .For example in Figure2,
df(a) = 2.
Chim et al. [7] also proposed that simply ignoring
the stopwords becomes impractical in STC, because
530
 suffix tree model is trying to keep the sequential
order of each words in a document, and the same
phrase or words might occur in different nodes of the
suffix tree. Based on the idea, they proposed the
definition of "stopnode", which applies the same idea
of stopwords in the suffix tree similarity measure
computation. A threshold idfthd of inverse document
frequency (idf ) is given to identify whether a node
is a stopnode. The experiment showed that the design
is very efficient.
In our paper, we adopt the same idea of
"stopnode" instead of stopwords, which also prove to
be efficient.
5. Experimental results
5.1. Data preparation
We prepare TDT [1] corpus for experiments,
which are benchmarks for event detection. We
choose TDT2 English corpus to run experiments. The
TDT2 English corpus contains news data collected
daily from 6 news sources, over a period of six
months. Detection is an unsupervised classification
task that does not involve training data, so all the
English corpus is used as evaluation data. More
details of our dataset are listed in Tablet.
Table 1. Details of dataset
Time January - June, 1998
Number of Articles 1930
Number of
Topics(Events) 70
Average articles
per event 27
5.2. Evaluation measures
TDT project has its own evaluation plan, i.e.,
detection performance is characterized in terms of the
probability of miss and false alarm errors, and these
error probabilities are then combined into a single
detection cost by assigning costs to miss and false
alarm errors. However, their tasks are not consistent
with ours. Thus, we choose the same evaluation
metrics as that in Yang et al.[2].Table2 illustrates the
contingency table for a cluster -event pair, where a,
b, c, d are document counts in the corresponding
531
Authorized licensed use limited to: SATHYABAMA UNIVERSITY. Downloaded on November 5, 2008 at 03:37 from IEEE Xplore. Restrictions apply.
cells. Five evolutional measures are defined
including Miss, False alarm, Recall, Precision, F-
measure. To measure global performance, two
averaging methods are used: micro-average by
summing the corresponding cells and then compute
the five measures, macro-average by averaging the
five measures of all events.
Table 2. Contingency table
in event not in event
in cluster a b
not in cluster c d
In our paper, we use micro-average of F-measure
and macro-average of F-measure as our evaluation
measures. We compute micro-average and macro-
average of F-measure score for each clustering result
respectively. F-measure is originally defined by C.J.
Rijsbergen [9], which is the harmonic mean of recall
and precision. The measure is defined as following
if(a + b + c) > 0, otherwise undefined:
F 2 * Precision * Recall 2a
Precision + Recall 2a + h + c
Where Precision = a l(a + b) if a+b>O ,
otherwise undefined; Recall = a l(a + c) if a + c > 0,
otherwise undefined.
5.3. Performance on the data set
To compare our approach with other algorithms,
Yang et al.'s augmented GAC and the generally used
KNN algorithm are chosen as baselines. Since GAC
is a hierarchical clustering method, we stop after
there are k clusters left, and run re-clustering 5 times
as the recommended settings in [2].For KNN we
report the results under the best threshold.
The original STC algorithm selects the 500 highest
scoring base clusters for further cluster merging, but
only the top 10 clusters are selected from the merged
clusters as the final clustering result. Thus we also
allowed GAC and KNN to generate 10 clusters in our
experiments to conduct as fair as possible
comparisons.
We use Java to implement all there algorithms, and
use Excel to plot all figures. Figurel illustrates the
results of the three approaches. The comparison of
the three methods is shown as Figure 1 which shows
 the performance of the three and Figure2 which
shows the execution time of the three.
0. .
)| !i ,
l
acavF
Figure3. The performance of the three methods
4 http://www.nist.gov/speech/tests/tdt/.
| GAC £TC
Figure4. The execution time of the three methods
(Time/s)
The best results obtained by STC are obviously.
We believe the main reason for STC's best
performance is the STC model, which is most
suitable for event detection. Furthermore, STC is the
fastest algorithm among the three, which own to the
direct-inserting policy of STC.
6. Conclusions and future work
Our work presented in this paper is mainly focused
on improving the effectiveness of document
clustering algorithms which is the hotspot of Event
detection. Efficiency optimization of the algorithm
has been a target of our current work. STC is a
suitable algorithm for clustering in event detection,
which has excellent features such as linear time
complexity and incrementality 1. In this work we
have shown a new event detection algorithm based
on improved STC algorithm. We improved STC in
the following aspects: improving method of merging
base clusters, new definition of cluster score, new
cluster labels and implementation of "stopnode". We
532
Authorized licensed use limited to: SATHYABAMA UNIVERSITY. Downloaded on November 5, 2008 at 03:37 from IEEE Xplore. Restrictions apply.
also demonstrate a substantial results using improved
STC algorithm, which indicate best performance and
shortest execution time compared to KNN and GAC.
In the course of this work, we encountered a
number of interesting questions and hope to answer
them in our future research. For one, we are not
satisfied with the threshold, although STC is not very
sensitive to the threshold. Second, we will consider
improving STC algorithm by combination with other
algorithms, which may obtain better results.
The present work can be extended in a number of
important directions. One is dictated by the multi-
lingual nature of TDT: STC algorithm should be
capable of dealing with documents in multiple
languages. We are very interested in implementing
the improved STC algorithm into Chinese.
Reference
[l]Topic detection and tracking (tdt) project. homepage:
[2]Y. Yang and J. G. Carbonell et al. Learning Approaches
for Detecting and Tracking News Events [J]. IEEE
Intelligent Systems: Special Issue on Applications of
Intelligent Information Retrieval, 1999, 14 (4):32 - 43.
[3]Y. Yang and J.Z. et al. Topic-conditioned novelty
detection In Proc. of SIGKDD international conference on
knowledge discovery and data mining,2002.
[4]J. Allan. Topic detection and tracking: event-based
information organization [M].Dordrecht: Kluwer Academic
Publishers, 2002.
[5]0. Zamir and 0. Etzioni. Web Document Clustering: A
Feasibility Demonstrate. In Proc. of SIGIR'98, University
of Washington, Settle, USA, 1998.
[6]Z. Lei and L. Wu et al. Incremental K-means Method
Based on Initialisation of Cluster Centers and Its
Application in News Event Detection. Journal of the China
Society for Scientific.25 (3):289-295, 2006.
[7]H. Chim and X. Deng. A New Suffix Tree Similarity
Measure for Document Clustering. In Proc. of WWW'2007,
Banff, Alberta, Canada, 2007.
[8]S. Branson and A. Greenberg. Clustering Web Search
Results Using Suffix Tree Methods. homepage:
http://stanford.edu/lass/archive/cs/cs276a/cs276a
[9]C. J. van Rijsbergen, Information Retrieval [M].London:
Butterworths, 1979.
[10]J.W.Yang. A Chinese Web Page Clustering Algorithm
Based on the Suffix Tree. Wuhan University Journal of
National Sciences [M]. 9 (5):817-822, 2004.