A DUAL FRAMEWORK AND ALGORITHMS FOR TARGETED ONLINE DATA DELIVERY

Abstract We present an alternative and more flexible approach that maximizes user utility by satisfying all users. It does this while minimizing the usage of system resources. We discuss the benefits of this latter approach and develop an adaptive monitoring solution Satisfy User Profiles (SUPs). Through formal analysis, we identify sufficient optimality conditions for SUP. Using real (RSS feeds) and synthetic traces, we empirically analyze the behavior of SUP under varying conditions. Our experiments show that we can achieve a high degree of satisfaction of user utility when the estimations of SUP closely estimate the real event stream, and has the potential to save a significant amount of system resources. We further show that SUP can exploit feedback to improve user utility with only a moderate increase in resource utilization.

Existing system A variety of emerging online data delivery applications challenge existing techniques for data delivery to human users, applications, or middleware that are accessing data from multiple autonomous servers. The first approach, most commonly used nowadays, maximizes user utility under the strict setting of meeting a priori constraints on the usage of system resources. Much of the existing research in pull-based data delivery casts the problem of data delivery as follows: Given some set of limited system resources, maximize the utility of a set of user profiles. We refer to this problem as OptMon1. To address some of the limitations of OptMon1, we propose a framework where we consider the dual of the previous optimization problem as follows: Given some set of user profiles, minimize the consumption of system resources while satisfying all user profiles.

Disadvantage

A Grid performance monitor tracks computational resources and notifies users of changes in system load and availability. Excessive probing of these machines may increase their load and hurt their performance. Clearly, minimizing the number of probes to such a source is important to keep probing costs low.

Proposed system The diversity of data sources and Web services currently available on the Internet and the computational Grid, as well as the diversity of clients and application requirements, poses significant infrastructure challenges. In this paper, we address the task of targeted data delivery. Users may have specific requirements for data delivery, e.g., how frequently or under what conditions they wish to be alerted about update events or update values, or their tolerance to delays or stale information. The challenge is to deliver relevant data to a client at the desired time, while conserving system resources. We consider a number of scenarios including RSS news feeds, stock prices and auctions on the commercial Internet, and scientific data sets and Grid computational resources. We consider an architecture of a proxy server that is managing a set of user profiles that are specified with respect to a set of remote autonomous servers.

With this class of problems, user needs are set as the constraining factor of the problem, while resource consumption is dynamic and changes with needs. We present an optimal algorithm in the OptMon2 class, namely, Satisfy User Profiles (SUPs). SUP is simple yet powerful in its ability to generate optimal scheduling of pull requests. SUP is an online algorithm; at each time point, it can get additional requests for resource monitoring. Through formal analysis, we identify sufficient conditions for SUP to be optimal given a set of updates to resources. ADVANTAGE Excessive probing of these machines may increase their load and hurt their performance. Clearly, minimizing the number of probes to such a source is important to keep probing costs high.

SOFTWARE HARDWARE REQUIREMENTS

HARDWARE REQUIREMENT CPU type Clock speed Ram size Hard disk capacity Monitor type Keyboard type : Intel Pentium 4 : 3.0 GHz : 512 MB : 40 GB : 15 Inch color monitor : internet keyboard

SOFTWARE REQUIREMENT Operating System: Windows XP Front End Back End : JAVA : SQL SERVER

Documentation MODULES User Profiles Notifications

Ms-Office

Execution Intervals and Monitoring Schedules and the Utility of Probing Sup optimality Sup algorithm OptMon2

MODULE DESCRIPTION Collecting User Profiles Profiles are declarative user specifications for data delivery. A profile should be easy to specify and sufficiently rich to capture client requirements. A profile should have clear semantics and be simple to implement. We assume that every resource r 2 R has a unique identifier (e.g., URI) and can be described using some schema (e.g., Relational Schema, DTD, XML-Schema, RDF-Schema, etc.) A resource can be queried using a suitable query language.

Notify user needs Clients use notification rules to describe their data needs and express the utility they assign with data delivery. A notification rule extends the Event-Condition- Action (ECA) structure in active databases and can be modified dynamically by the user. A notification rule is a quadruple hQ; Tr; T ; Ui.Qis a query written in some suitable query language. T is the epoch in which rules are evaluated. Finally, U is a utility expression specifying the utility client gains by notifications of Q. A notification query Q is specified over a set of resources from the profile domain denoted by Domain; . Queries are equivalent to actions in the ECA structure.

Execution Intervals and Monitoring Once an event, specified in the trigger part of the notification rule, occurs, the trigger condition is immediately evaluated and if it is true, the notification is said to be executable. The period in which a notification rule is executable was referred to in the literature as life. We emphasize here the difference between the executable period of a notification (life) and the period in which rules, in general, can be evaluated (epoch). In which an update is available for monitoring only until the next update to the same resource occurs. Schedules and the Utility of Probing In each execution interval, every resource referenced by s query Q is probed at least once. It is worth noting that each execution interval E () is associated with some (either update or periodical) event, and therefore, a schedule that satisfies the notification rule actually needs to capture every event required in . We shall assume the use of a binary utility, i.e., w 1. Examples of strict utility functions include uniform (where utility is independent of delay) and sliding window (where utility is 1 within the window and 0 outside it). Examples of nonstrict utility functions are linear and nonlinear decay functions. Nonstrict utilities quantify tolerance toward delayed data delivery (or latency). We shall restrict ourselves in this work to strict utility functions. The case of nonstrict utility functions can be handled in the scope of OptMon2 problems by allowing users to define a threshold for the minimal utility required in the user profile

Sup optimality Probing at the last possible chronon ensures an optimal usage of system resources (probes) while still satisfying user profiles. However, due to the stochastic nature of the process, probing later may decrease the probability of satisfying the profile. This is true, for example, with hard deadlines where once the deadline has passed, the utility is 0. Determining an optimal chronon for probing, i.e., the one that maximizes the probability of satisfying the profile, depends on the stochastic process of choice, and is itself an interesting optimization problem. Identify best by Sup algorithm

The main intuition behind the SUP algorithm is to identify the best candidate chronon in which the assignment of probes to resources maximizes the number of execution intervals that can benefit from each probe. This will then lead to a reduction of the number of resources in QI I that need to be probed during each execution interval I. We identify the best candidate chronons by delaying the probes of execution intervals to the last possible chronon in which the utility is still positive Implementation of OptMon2 We propose a dual formulation OptMon2, which reverses the roles of user utility and system constraints, setting the fulfillment of user needs as the hard constraint. OptMon2 assumes that the system resources that will be consumed to satisfy user profiles should be determined by the specific profiles and the environment, e.g., the model of updates, and does not assume an a priori limitation of system resources. OptMon2 can be stated as the following general formulation: minimize system resource usage satisfying user profiles:

System Flow Diagram

LOGIN

USER PROFILE

NOTIFICATION

EXECUTION MONITORING

SCHDULE PROBING

SUP ALGORITHM

OPTIMIZING

DATA SENDING

CONCLUSIONS
We focused on pull-based data delivery that supports user profile diversity. Minimizing the number of probes to sources is important for pull-based applications to conserve resources and improve scalability. Solutions that can adapt to changes in source behavior are also important due to the difficulty of predicting when updates occur. We have addressed these challenges through the use of a new formalism of a dual optimization problem (OptMon2), reversing the roles of user utility and system resources. This revised specification leads naturally to a surprisingly simple, yet powerful algorithm (SUP) which satisfies user specifications while minimizing system resource consumption. We have formally shown that SUP is optimal for OptMon2 and under certain restrictions can be optimal for OptMon1 as well. Using RSS data traces as well as synthetic data, that SUP can satisfy user profiles and capture more updates compared to existing policies. SUP is adaptive and can dynamically change monitoring schedules. Our experiments show that using feedback in SUP improves the performance with a moderate increase in the number of needed probes. We believe that the main impact of this work will be in what is now known as the Internet of things, where sensor data are collected, analyzed, and utilized in many different ways, based on users needs. With the Internet of things, user profiles, and their satisfaction dictate the way data are utilized, and monitoring sensor data efficiently is a mandatory prerequisite to the creation of any information system that is based on such data. OptMon2 is defined in such a way that satisfaction of a user profile is a hard constraint.