Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/boinc/boinc-combined-stats


https://github.com/boinc/boinc-combined-stats

Last synced: about 2 months ago
JSON representation

Awesome Lists containing this project

README 

        
This SVN repository contains the source code for the BOINC Combined Statistics web site

(http://boinc.netsoft-online.com/). 



Here is a quick guide to the components of the repository:



bsimport - This directory contains the java source code of the import and 

           statistic calculation program. This is the program that is run

           daily to import the statistics from the various BOINC projects

           into the stats database.



database - This directory contains the MySQL table creation and update scripts

           to create the underlying database for the statistics.



e107     - This directory contains the theme and plugin modules for the e107

           web site (http://www.e107.org). When these are added and activated

           to an e107 system, will produce the viewable results of the stats.

           The site currently uses version 0.7.8 of e107.



scripts  - This directory contains other various scripts used by BCS.



The general flow of how BOINC Combined Statistics runs is:



1) The fetch_files.sh script runs twice a day to grab the export files

   from various projects. The scheduled time of the run is set so it

   should finish fetching the files before the import process runs.

   It utilizes wget, and only downloads the file if it has changed from

   the previous time the script ran.  The idea behind running the fetch

   multiple times is to avoid temporary connection failures to projects.



2) The bsimport java application runs. It runs once a day. This application

   will take the data files from each project and import the data into

   the mysql database. After a sucessful import, it creates a backup file

   of the data. If any of the three import files (team,host,user) has a

   problem, the program will go through and decay the RAC values so all 

   values in the DB are calculated to the same time.



   There are a number of optimizations in the import process. For example,

   if the host hasn't been heard from in some time, and its RAC is already

   decayed to zero, then the import process just skips past it - not doing

   any database transactions for that particular host.



   Once all the data has been imported, bsimport creates the stats. It runs

   through and calculates things like world rank, etc. Often these are 

   done via SQL statments, letting the DB do the work. Other times - in 

   particular the credit per cpu second analysis, the program does the

   full calculation as it can be done far faster by reading all the host

   data in order of host cpid than the database can do it.



   After the stats are calculated, bsimport exports a sorted list of the

   combined user data (currently only user data, not host or team data) 

   for use by others to import. The file is a gzip'd xml file that is

   made available for download on the web site.



  

3) The statistics are made available via the e107 plugins and other various

   web RPC pages. These web pages access the data in the mysql database for

   display via XML, web pages and graphs.