Skip to content

Commit

Permalink
(no bug) improve vagrant install flow
Browse files Browse the repository at this point in the history
  • Loading branch information
rhelmer committed May 31, 2014
1 parent d78ed26 commit e8e8cd9
Show file tree
Hide file tree
Showing 3 changed files with 48 additions and 38 deletions.
14 changes: 6 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,8 @@ test-socorro: bootstrap
$(ENV) $(PG_RESOURCES) $(RMQ_RESOURCES) $(ES_RESOURCES) PYTHONPATH=$(PYTHONPATH) $(COVERAGE) run $(NOSE)
$(COVERAGE) xml

# makes thing semantically consistent (test-{component}) while avoiding
# building the webapp twice to save a little time
test-webapp: webapp-django
# alias to webapp-django
test-webapp:
cd webapp-django; ./bin/jenkins.sh

bootstrap:
git submodule update --init --recursive
Expand All @@ -54,15 +52,15 @@ bootstrap:
$(VIRTUALENV)/bin/pip install tools/peep-1.1.tar.gz
$(VIRTUALENV)/bin/peep install --download-cache=./pip-cache -r requirements.txt

install: bootstrap reinstall
install: bootstrap bootstrap-webapp reinstall

# this a dev-only option, `make install` needs to be run at least once in the checkout (or after `make clean`)
reinstall: install-socorro
# record current git revision in install dir
git rev-parse HEAD > $(PREFIX)/application/socorro/external/postgresql/socorro_revision.txt
cp $(PREFIX)/stackwalk/revision.txt $(PREFIX)/application/socorro/external/postgresql/breakpad_revision.txt

install-socorro: webapp-django
install-socorro: bootstrap-webapp
# package up the tarball in $(PREFIX)
# create base directories
mkdir -p $(PREFIX)/application
Expand Down Expand Up @@ -112,8 +110,8 @@ json_enhancements_pg_extension: bootstrap
# every time Socorro is built
if [ ! -f `pg_config --pkglibdir`/json_enhancements.so ]; then sudo env PATH=$$PATH $(VIRTUALENV)/bin/python -c "from pgxnclient import cli; cli.main(['install', 'json_enhancements'])"; fi

webapp-django: bootstrap
cd webapp-django; ./bin/jenkins.sh
bootstrap-webapp:
cd webapp-django; ./bin/bootstrap.sh

stackwalker:
# Build JSON stackwalker
Expand Down
3 changes: 1 addition & 2 deletions config/apache.conf-dist
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
NameVirtualHost *:80
WSGIPythonPath /data/socorro/application
WSGIPythonHome /data/socorro/socorro-virtualenv:/data/socorro/webapp-django/virtualenv
WSGIPythonPath /data/socorro/application:/data/socorro/socorro-virtualenv/lib/python2.6/site-packages/:/data/socorro/webapp-django/virtualenv/lib/python2.6/site-packages

# Django app, public web UI
<VirtualHost *:80>
Expand Down
69 changes: 41 additions & 28 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -237,14 +237,21 @@ To run and hack on Socorro apps, you will need:

3) The Django web service and its tests need the LESS preprocessor to be
installed and on your $PATH
You may need to run this as the *root* user depending on how node.js was
installed
::
npm install -g less

Socorro can install the dependencies into a virtualenv for you, then
just activate it and set your PYTHONPATH

Socorro can install the dependencies into a virtualenv for you.
You only need to run this once
::
export PATH=$PATH:/usr/pgsql-9.3/bin/
make bootstrap
make bootstrap bootstrap-webapp

Before running any Socorro components, always make sure that the virtualenv
is activated and the PYTHONPATH is set
::
. socorro-virtualenv/bin/activate
export PYTHONPATH=.

Expand All @@ -255,15 +262,18 @@ virtualenwrapper.
Add a new superuser account to PostgreSQL
-----------------------------------------

Create a superuser account for yourself, and one for running tests:
As the *root* user:
Create a superuser account for yourself. Make sure to put your username
and desired password instead of YOURNAME and YOURPASS.
As the *postgres* user:
::
su - postgres -c "createuser -s $USER"
psql template1 -c \
"create user YOURNAME with encrypted password 'YOURPASS' superuser"

For running unit tests, you'll want a test user as well (make sure
to remove this for production installs):
to remove this for production installs).
::
psql template1 -c "create user test with password 'aPassword' superuser"
psql template1 -c \
"create user test with encrypted password 'aPassword' superuser"

Also, before you run unit tests or make, be sure to copy and edit this file:

Expand Down Expand Up @@ -305,7 +315,7 @@ Run unit/functional tests

From inside the Socorro checkout
::
make test
make test test-webapp


Install stackwalker
Expand Down Expand Up @@ -334,12 +344,7 @@ like, you also have the option to generate and populate the DB with synthetic
test data
::
./socorro/external/postgresql/setupdb_app.py --database_name=breakpad \
--fakedata --dropdb --database_superusername=$USER

Or, run setupdb_app.py to create an empty breakpad database and load the schema:
::
./socorro/external/postgresql/setupdb_app.py --database_name=breakpad \
--database_superusername=$USER
--fakedata --dropdb

IMPORTANT NOTE - many reports use the reports_clean_done() stored
procedure to check that reports exist for the last UTC hour of the
Expand All @@ -348,10 +353,6 @@ volume does not guarantee one crash per hour, you may want to modify
this function in
socorro/external/postgresql/raw_sql/procs/reports_clean_done.sql
and reload the schema
::

./socorro/external/postgresql/setupdb_app.py --database_name=breakpad \
--dropdb --database_superusername=$USER


Create partitioned reports_* tables
Expand All @@ -374,26 +375,29 @@ Copy default config files
cp config/processor.ini-dist config/processor.ini
cp config/middleware.ini-dist config/middleware.ini
cp webapp-django/crashstats/settings/local.py-dist \
webapp-django/crashstats/settings/local.py
webapp-django/crashstats/settings/local.py

You may need to edit these config files - for example collector (which is
generally a public service) might need listen on the correct IP address.

By default they listen on localhost only.
By default they listen on localhost only, which should be fine for local
development.

Run Socorro services using Honcho (configured in Procfile)
::
honcho start

You can also start individual services:
Alternatively you can also start individual services:
::
honcho start web
honcho start collector
honcho start middleware
honcho start processor

If you want to modify something that is common across config files like PostgreSQL username/hostname/etc, refer to config/common_database.ini-dist and the "+include" line in the service-specific config files (such as collector.ini
and processor.ini). This is optional but recommended.
If you want to modify something that is common across config files like
PostgreSQL username/hostname/etc, refer to config/common_database.ini-dist and
the "+include" line in the service-specific config files (such as
collector.ini and processor.ini). This is optional but recommended.

.. _systemtest-chapter:

Expand Down Expand Up @@ -455,9 +459,10 @@ As the *root* user:
chown apache /home/socorro/primaryCrashStore /home/socorro/fallback
chmod 2775 /home/socorro/primaryCrashStore /home/socorro/fallback

Ensure that the user doing installs owns the install dir:
Ensure that the user doing installs owns the install dir,
as the *root* user:
::
su -c "chown $USER /data/socorro"
chown socorro /data/socorro

Install socorro
---------------
Expand All @@ -479,9 +484,14 @@ Install configuration to system directory

From inside the Socorro checkout, as the *root* user
::
cp config/*.ini-dist /etc/socorro
cp config/\*.ini-dist /etc/socorro

Make sure the copy each .ini-dist file to .ini and configure it.

Make sure the copy each *.ini-dist file to *.ini and configure it.
In particular, you must change the web server in collector.ini
and middlware.ini to support Apache mod_wsgi rather than the standalone
server::
wsgi_server_class='socorro.webapi.servers.ApacheModWSGI'

It is highly recommended that you customize the files
to change default passwords, and include the common_*.ini files
Expand Down Expand Up @@ -549,9 +559,12 @@ edit /data/socorro/webapp-django/crashstats/settings/local.py:
DEV = False
COMPRESS_OFFLINE = True
SECRET_KEY = '' # set this to something unique
# adjust this for your site!
ALLOWED_HOSTS = ['crash-stats.example.com']

Allow Django to create the database tables it needs for managing sessions:
::
. /data/socorro/webapp-django/virtualenv/bin/activate
/data/socorro/webapp-django/manage.py syncdb --noinput

Copy the example Apache config into place from the Socorro checkout as the
Expand Down

0 comments on commit e8e8cd9

Please sign in to comment.