LargeDataSets ProjectIdeas 2011 12

From Wiki
Revision as of 19:26, 19 January 2012 by MThomas (Talk | contribs) (Interesting Data Sets for Spring 2012 Projects)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Project Ideas for Spring 2012

  1. LSA analysis of any of the smaller data sets below, feed the results to a spectral clustering group working with Dr Carter
  2. work on analysis, in R or Matlab or with python programs, of astronomical data (with Dr DeVries)
  3. Straightforward operating system comparison: MySQL vs MySQL, same hardware. Pick two operating systems, run the same set of queries over the same data set (generated by Fall 2011 data generating program students wrote?). Which is faster, or is there no difference? We could run tests on the netbooks, or we might hang an 'out of order' sign on one of the CS lab machines for a little while. (Early on in the Spring semester, or before the semester. Not during Spring finals.) If there is a difference, can we figure out why the operating systems cause performance differences in the DBMS?
  4. Straightforward DBMS comparison: MySQL vs PostgreSQL, same hardware, same data, same set of queries. Which DBMS should a user wanting to cope with Census data use? (Suppose the Census Bureau, for example, needs to install a DBMS.)
  5. Validate our data generation program. Load real Census data into DBMS; load data from student data generation program.

Interesting Data Sets for Spring 2012 Projects

  1. Census data, 1990, 2000, 2010
  2. American Community Survey Data
  3. any data set with spatial data can be mapped and compared to Census Data
    1. FBI Uniform Crime Data?
    2. FCC broadband access data
  4. Spitzer/Fitzer/Pitzer? data, stars, Dr DeVries - looking for young stars.
  5. Atacoma (sp?) Large Millimeter array, Dr DeVries - looking for signs of collapsing gas (data not really available yet - late Spring 2012)
  6. Data from the CSU Stanislaus Institutional Research Office
    1. CSU Stan has test results from 6 years of WP exams
    2. Nat'l Survey of Student Engagement data, 6 years worth, and about 2 years of Faculty Survey of Student Engagement data
    3. Dept of Education 'California Basic Educational System' (CBEdS) database. Can be organized by county / district / school to look at the student population that "feeds" CSU Stanislaus
      1. web site where CBEdS / CALPADS data files may be downloaded
    4. other data sets also available
  7. Usenet flame data (Dr Martin)
  8. medical web pages, annotated (Dr Martin)
  9. group communication, annotated with effectiveness of outcomes of communication (Dr Martin)


Safecast offers data (under Creative Commons license) gathered by volunteers in Japan, on radiation levels in Japan, gathered by people who put Safecast sensors on their cars and drive around taking readings. (Also, they have some fixed sensors.)