forked from quandl/quandl-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Feature ap 243/v3 client (quandl#37)
* new quandl client using v3 api * send up request source and version * allow passing in file name, and allow passing in download type as query param * Update readme docs for upgrading and data tables. * Increase the versions of python we test against on CI and tox. Remove .get from models to avoid 2x call to the api. Fix some issues where _raw_data was not named correctly. Remove version 3.2 from travis as it fails even though it should not. * data tables for developers * Fix a few flake8 issues and run it locally with tox. * Improve the connection error handling code based on suggestion from community member femtotrader. * fix file format * Improve upgrade documentation to take into account pickled versions of api_key. * Notes based on feedback. * analysts get and bulkdownload methods * refactoring get_table * using python bool and remove returns options * remove user warning * Update README.md * Reduce memory usage and processing time by not using ordered-dictionaries for the data. Update the way nested dates are detected and converted into real dates. Also update tests to match. Add jsondate to tox dependencies. Ensure that tox installs same dependencies as setup.py * Updates based on feedback. * MergedDataset: dont make metadata calls when getting data * Update FOR_ANALYSTS.md add bulkdownload description some updates * Missing package in setup. * release setup * messages * Update 2_SERIES_UPGRADE.md * ready and message updates * Update README.md minor change to test branch * add changelog
- Loading branch information
Showing
70 changed files
with
3,308 additions
and
457 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,8 @@ | ||
|
||
*.pyc | ||
|
||
/.tox/ | ||
/.eggs | ||
/quandl.egg-info | ||
/build | ||
/dist | ||
/tmp/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
sudo: false | ||
language: python | ||
python: | ||
- "2.7" | ||
- "3.3" | ||
- "3.4" | ||
- "3.5" | ||
install: | ||
- pip install flake8 | ||
- pip install -r requirements.txt | ||
script: | ||
- flake8 | ||
- python -W always setup.py -q test | ||
cache: pip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# 2.x Series package Notes | ||
|
||
With the release of version 3 of our API we are officially deprecating version 2 of the Quandl Python package. We have re-written the package and will be moving forward with a 3.x.x package under the new namespace of `quandl` that will rely on version 3 of our RESTful API. | ||
|
||
## Upgrading | ||
|
||
There are numerous advantages to upgrade from the older 2.x series package including improved performance and stability. The upgrade process is fairly simple. | ||
|
||
Upgrade your package using pip and running: `pip install --upgrade quandl` | ||
|
||
Wherever you have: | ||
|
||
```python | ||
import Quandl | ||
``` | ||
|
||
change this to: | ||
|
||
```python | ||
import quandl as Quandl | ||
``` | ||
|
||
Additionally if you were relying on a saved version of your `api key` _(authtoken)_ in your scripts please note that this functionality has been dropped from the existing package. To continue to load your api key from the filesystem we recommend you import the key and set it via the api config `quandl.ApiConfig.api_key`. Below is a example of importing your previously pickled key if it exists: | ||
|
||
```python | ||
import quandl as Quandl | ||
import pickle | ||
import os.path | ||
if os.path.isfile('authtoken.p'): | ||
Quandl.ApiConfig.api_key = pickle.load(open('authtoken.p', 'rb')) | ||
``` | ||
|
||
That's it! The new package should be ready to use. | ||
|
||
## Continuing with the Old Release | ||
|
||
If you wish to continue using the old package during this transitional period please follow the instructions below: | ||
|
||
https://github.com/quandl/quandl-python/tree/v2.8.9 | ||
|
||
To continue using Quandl API version 2, do the following: | ||
|
||
1. Ensure you have [pip installed](https://pip.pypa.io/en/latest/installing.html) | ||
|
||
2. In your Python program's directory, execute `pip freeze > requirements.txt`. Alternatively, create the `requirements.txt` file and enter the desired Quandl package version, e.g., `Quandl==2.8.8`. | ||
|
||
3. Execute `pip install -r requirements.txt` to ensure the desired Quandl package version is installed. | ||
|
||
Note: for the sample code to work you must have access to read the file with the auth token. The sample script also assumes that file is stored in the same directory from where the script is run. If you wish to use a key in another directory you will need to match the path used in the sample. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
### 3.0.0 - 2016-04-22 | ||
|
||
* Datatables for developers | ||
* Datatables for analysts | ||
* Backwards compatibility | ||
* Documentation updates |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
# Quick Method Guide - Quandl-Python | ||
|
||
This quick guide offers convienent ways to retrieve individual datasets or datatables with the Python package without the need for complex commands. | ||
|
||
## Retrieving Data | ||
|
||
Retrieving data can be achieved easily using the two methods `quandl.get` for datasets and `quandl.get_table` for datatables. In both cases we strongly recommend that you set your api key via: | ||
|
||
```python | ||
import quandl | ||
quandl.ApiConfig.api_key = 'tEsTkEy123456789' | ||
``` | ||
|
||
### Datasets | ||
|
||
To retrieve a Quandl dataset you can make the following call: | ||
|
||
```python | ||
import quandl | ||
data = quandl.get('NSE/OIL') | ||
``` | ||
|
||
This finds all data points for the dataset `NSE/OIL` and stores them in a pandas dataframe. You can then view the dataframe with: | ||
|
||
```python | ||
data.head() | ||
``` | ||
|
||
However we recommend applying filters to streamline the results. To do this you may want to specify some additional filters and transformations. | ||
|
||
```python | ||
import quandl | ||
data = quandl.get('NSE/OIL', start_date='2010-01-01', end_date='2014-01-01', | ||
collapse='annual', transformation='rdiff', | ||
rows=4) | ||
``` | ||
|
||
This revised query will find all data points annually for the dataset `NSE/OIL` between the year 2010 and 2014 and transform them using the `rdiff` transformation. The query parameters used are documented in our [api docs](https://www.quandl.com/docs/api#data). Since the `data` was retrieved in the pandas format you can then view the dataframe with data.head(). | ||
|
||
#### Available parameters: | ||
|
||
The following additional parameters can be specified for a dataset call: | ||
|
||
| Option | Explanation | Example | Description | | ||
|---|---|---|---| | ||
| api_key | Your access key | `api_key='tEsTkEy123456789'` | Used to identify who you are and provide more access. Only required if not set via `quandl.ApiConfig.api_key=` | | ||
| \<filter / transformation parameter\> | A parameter which filters or transforms the resulting data | `start_date='2010-01-01` | For a full list see our [api docs](https://www.quandl.com/docs/api#data) | | ||
|
||
For more information on how to use and manipulate the resulting data see the [pandas documentation](http://pandas.pydata.org/). | ||
|
||
#### Download Entire Database (Bulk Download) | ||
|
||
You can download all the data in a database in a single call. The following will download the entire EOD database as a zip file to your current working directory: | ||
|
||
```python | ||
import quandl | ||
quandl.bulkdownload('EOD') | ||
``` | ||
|
||
After the download is finished, the `quandl.bulkdownload` will return the filename of the downloaded zip file. | ||
|
||
To download database data from the previous day, use the download_type option: | ||
|
||
```python | ||
import quandl | ||
quandl.bulkdownload('EOD', download_type='partial') | ||
``` | ||
|
||
You can also change the filename of the downloaded zip file by using the filename option: | ||
|
||
```python | ||
import quandl | ||
quandl.bulkdownload('EOD', filename='/my/path/EOD_DB.zip') | ||
``` | ||
|
||
#### Download Multiple Codes | ||
|
||
Sometimes you want to compare two codes. For example if you wanted to compare the closing prices for Apple and Microsoft, you would obtain the two Quandl codes: | ||
|
||
`GOOG/NASDAQ_AAPL` vs. `GOOG/NASDAQ_MSFT` | ||
|
||
Append the column you wish to get with a `.`, and put them into an array. | ||
|
||
`['GOOG/NASDAQ_AAPL.4','GOOG/NASDAQ_MSFT.4']` | ||
|
||
Just make a normal get call with the array passed to get, and your multiset will be returned. | ||
|
||
#### Download Multiple Codes (Multiset) | ||
|
||
If you want to get multiple codes at once, delimit the codes with ',', and put them into an array. This will return a multiset. | ||
|
||
For example: | ||
```python | ||
data = quandl.get(['GOOG/NASDAQ_AAPL.4','GOOG/NASDAQ_MSFT.4']) | ||
``` | ||
|
||
Which outputs: | ||
|
||
``` | ||
See www.quandl.com/docs/api for more information. | ||
Returning Dataframe for ['GOOG.NASDAQ_AAPL.4', 'GOOG.NASDAQ_MSFT.4'] | ||
GOOG.NASDAQ_AAPL - Close GOOG.NASDAQ_MSFT - Close | ||
Date | ||
1997-08-20 6.16 17.57 | ||
1997-08-21 6.00 17.23 | ||
1997-08-22 5.91 17.16 | ||
1997-08-25 5.77 17.06 | ||
1997-08-26 5.56 16.88 | ||
``` | ||
|
||
### Datatables | ||
|
||
Datatables work similarly to datasets but provide more flexibility when it comes to filtering. For example a simple way to retrieve datatable information would be: | ||
|
||
```python | ||
import quandl | ||
data = quandl.get_table('ZACKS/FC') | ||
``` | ||
|
||
Given the volume of data stored in datatables, this call will retrieve the first page of the `ZACKS/FC` datatable. You may turn on pagination to return more data by using: | ||
|
||
```python | ||
import quandl | ||
data = quandl.get_table('ZACKS/FC', paginate=True) | ||
``` | ||
|
||
This will retrieve multiple pages of data and merge them together as if they were one large page. In some cases, however, you will still exceed the request limit. In this case we recommend you filter your data using the available query parameters, as in the following example: | ||
|
||
```python | ||
import quandl | ||
data = quandl.get_table('ZACKS/FC', paginate=True, m_ticker=['AAPL', 'MSFT'], per_end_date={'gte': '2015-01-01'}}, qopts={'columns':['m_ticker', 'per_end_date']}) | ||
``` | ||
|
||
In this query we are asking for more pages of data, `m_ticker` values of either `AAPL` or `MSFT` and a `per_end_date` that is greater than or equal to `2015-01-01`. We are also filtering the returned columns on `m_ticker`, `per_end_date` and `comp_name` rather than all available columns. The output format is `pandas`. | ||
|
||
#### Available parameters: | ||
|
||
The following additional parameters can be specified for a datatable call: | ||
|
||
| Option | Explanation | Example | Description | | ||
|---|---|---|---| | ||
| api_key | Your access key | `api_key='tEsTkEy123456789'` | Used to identify who you are and provide more access. Only required if not set via `quandl.ApiConfig.api_key=` | | ||
| \<filter / transformation parameter\> | A parameter which filters or transforms the resulting data | `start_date='2010-01-01'` | For a full list see our [api docs](https://www.quandl.com/docs/api#datatables) | | ||
| paginate | Wether to autoamtically paginate data | `paginate=True` | Will paginate through the first few pages of data automatically and merge them together in a larger output format. | | ||
|
||
For more information on how to use and manipulate the resulting data see the [pandas documentation](http://pandas.pydata.org/). | ||
|
||
#### Things to note | ||
|
||
* Some datatables will return `sample` data if a valid api key is not used. If you are not recieving all of the expected data please double check your API key. | ||
* When using the paginate=True option depending on the total number of rows in the result set you may receive an error indicating that there are more pages that have not been downloaded. This is due to a very large result sets that would be too large to send via the analyst method. If this happens we recommend you take one of two approaches: | ||
* *(recommended)* Refine your filter parameters to retrieve a smaller results set | ||
* Use the the [Detailed](./FOR_DEVELOPERS.md) method to iterate through more of the data. | ||
|
||
## More usages | ||
|
||
For even more advanced usage please see our [Detailed Method Guide] (./FOR_DEVELOPERS.md). |
Oops, something went wrong.