= Upgrade, flask, sage 4.7, and ext4 filesystem =

People: W. Stein, J. Grout, M. Hansen

== Published Interacts ==

  * See if it is possible to get Mitesh Patel's patch to work: http://trac.sagemath.org/sage_trac/ticket/7908
  * Test server (might be up): http://sage.math.washington.edu:8976/pub/

== Database ==

  * If we stick with the filesystem: NEED a very fast filesystem that supports > 4 million inodes and > 32000 directories per directory, i.e., ext4.   This is difficult because boxen and sage.math are both Ubuntu 8.10.
  * Also, want to at a minimum have user and worksheet metadata stored in database so startup is faster. 
  * Could run the server in a virtual machine, which would support ext4.  But are virtual machines fast enough these days?  They weren't last year.  This would be worth trying. 
  * Or stick with the filesystem (see below).


== Filesystem Thoughts ==

  * (mhansen is doing this) Fix the {{{__worksheet}}} object to not load everything.  Scary stuff if you look at notebook.py.
  * (mhansen) Write a script to prune the filesystem, e.g., deleting cell directories that are empty.
  * (mhansen?) Make it so user directories are stored in a hierarchical way.  E.g., home/_store/


== Current Servers ==


  * http://sagenb.org
  * http://prep.sagenb.org
  * http://tutorial.sagenb.org
  * http://480.sagenb.org
  * http://alpha.sagenb.org
  * http://demo.sagenb.org
  * http://demo2.sagenb.org
  * http://nt.sagenb.org
  * http://standalone.sagenb.org
  * http://test.sagenb.org
  * http://uw.sagenb.org
  * http://flask.sagenb.org    -- not upgraded

== Backup Strategy ==

  ?

== Untrusted User Code: must all be executed in a virtual machine ==

  ? 


== Monitoring ==


  * I spent a day writing this script to *manage* a collection of notebook servers: [[attachment:admin]]

  * Nagios -- investigate for monitoring
  * Or extend this script: http://wstein.org/home/wstein/bin/watch.sage

== Virtual Machines ==

  * I tried install virtualbox 4 on sage.math and failed, due to the Linux source and Linux kernel being out of sync somehow.  Probably just needs an upgrade and reboot.

  * I am scared to upgrade virtualbox on boxen.math.

  * The Virtualbox on boxen.math is somewhat unstable and very old. 

== William's TODO List ==
{{{
 [x] official sagenb list of optional packages, and install them all.
 [x] make a "stop" script for servers that works.
 [x] fix any remaining links issues, then archive the fixlinks script
 [x] start servers and test
 [x] re-enable automatic backups
 [x] upgrade my sagenb install to include rado's fix for unicode issue, etc. 
 [x] search is broken (mhansen is on it)
 [x]  The _sage_ directory gets too big. (mhansen is on it)
 [x] script that computes and summarizes stats about all notebook servers:
      - number of users
      - how many users who evaluated code during the last day, week, etc. -- implement by reading last mod time on all history.pickle files.
      - what code was evaluated recently (read from history)
      - size of each notebook server
      - disk usage of each user
      - how many worksheets each user has
      
    (use a nosqlite db?)
    datetime server username last_datetime num_worksheets disk_usage

 [ ] make it so all pickle db's get saved in a non-corruptable way (atomic)
 [ ] push merged combined changes to wstein repo?
 [ ] MATHJAX (or jsmath image fonts ?!)
 [ ] twisted -- number of threads?



 [ ] ban copying results over if too big.


 [ ] somehow migrate flask.sagenb.org ???  or just wait.

}}}