V0.2.2 (microsoft#19)

* v0.2.2

separate the HPO part into the module flaml.tune
enhanced implementation of FLOW^2, CFO and BlendSearch
support parallel tuning using ray tune
add support for sample_weight and generic fit arguments
enable mlflow logging

Co-authored-by: Chi Wang (MSR) <[email protected]>
Co-authored-by: qingyun-wu <[email protected]>

.github/workflows/python-package.yml

@@ -38,6 +38,10 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install -e .[test]
+      - name: If linux or max, install ray
+        if: matrix.os == 'macOS-latest' or 'ubuntu-latest'
+        run: |
+          pip install -e .[ray]
       - name: Lint with flake8
         run: |
           # stop the build if there are Python syntax errors or undefined names

.gitignore

@@ -148,3 +148,4 @@ dmypy.json
 cython_debug/
 /catboost_info
 notebook/*.pkl
+notebook/.azureml

LICENSE

@@ -19,3 +19,22 @@
     LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
     OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
     SOFTWARE
+
+-------------
+Code in tune/[analysis.py, sample.py, trial.py] and 
+searcher/[suggestion.py, variant_generator.py] is adapted from
+https://github.com/ray-project/ray/blob/master/python/ray/tune/
+
+# Copyright 2020 The Ray Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

README.md

@@ -1,12 +1,17 @@
 # FLAML - Fast and Lightweight AutoML
 
+<p align="center">
+    <img src="https://github.com/microsoft/FLAML/raw/v0.2.2/docs/images/FLAML.png"  width=200>
+    <br>
+</p>
+
 FLAML is a Python library designed to automatically produce accurate machine
 learning models with low computational cost. It frees users from selecting
 learners and hyperparameters for each learner. It is fast and cheap.
 The simple and lightweight design makes it easy to extend, such as
-adding customized learners or metrics. FLAML is powered by a new, cost-effective
-hyperparameter optimization and learner selection method invented by
-Microsoft Research.
+adding customized learners or metrics. FLAML is powered by a new, [cost-effective
+hyperparameter optimization](https://github.com/microsoft/FLAML/tree/main/flaml/tune)
+and learner selection method invented by Microsoft Research.
 FLAML is easy to use:
 
 * With three lines of code, you can start using this economical and fast
@@ -23,10 +28,10 @@ tool for XGBoost, LightGBM, Random Forest etc. or a customized learner.
 automl.fit(X_train, y_train, task="classification", estimator_list=["lgbm"])
 ```
 
-* You can embed FLAML in self-tuning software for just-in-time tuning with
-low latency & resource consumption.
+* You can also run generic ray-tune style hyperparameter tuning for a custom function.
 ```python
-automl.fit(X_train, y_train, task="regression", time_budget=60)
+from flaml import tune
+tune.run(train_with_config, config={…}, init_config={…}, time_budget_s=3600)
 ```
 
 ## Installation
@@ -51,22 +56,22 @@ A basic classification example.
 ```python
 from flaml import AutoML
 from sklearn.datasets import load_iris
-# Initialize the FLAML learner.
+# Initialize an AutoML instance
 automl = AutoML()
-# Provide configurations.
+# Specify automl goal and constraint
 automl_settings = {
     "time_budget": 10,  # in seconds
     "metric": 'accuracy',
     "task": 'classification',
     "log_file_name": "test/iris.log",
 }
 X_train, y_train = load_iris(return_X_y=True)
-# Train with labeled input data.
+# Train with labeled input data
 automl.fit(X_train=X_train, y_train=y_train,
                         **automl_settings)
 # Predict
 print(automl.predict_proba(X_train))
-# Export the best model.
+# Export the best model
 print(automl.model)
 ```
 
@@ -75,35 +80,49 @@ A basic regression example.
 ```python
 from flaml import AutoML
 from sklearn.datasets import load_boston
-# Initialize the FLAML learner.
+# Initialize an AutoML instance
 automl = AutoML()
-# Provide configurations.
+# Specify automl goal and constraint
 automl_settings = {
     "time_budget": 10,  # in seconds
     "metric": 'r2',
     "task": 'regression',
     "log_file_name": "test/boston.log",
 }
 X_train, y_train = load_boston(return_X_y=True)
-# Train with labeled input data.
+# Train with labeled input data
 automl.fit(X_train=X_train, y_train=y_train,
                         **automl_settings)
 # Predict
 print(automl.predict(X_train))
-# Export the best model.
+# Export the best model
 print(automl.model)
 ```
 
-More examples: see the [notebook](https://github.com/microsoft/FLAML/tree/main/notebook/flaml_demo.ipynb)
+More examples can be found in [notebooks](https://github.com/microsoft/FLAML/tree/main/notebook/).
 
 ## Documentation
 
 The API documentation is [here](https://microsoft.github.io/FLAML/).
 
+Read more about the 
+hyperparameter optimization methods
+in FLAML [here](https://github.com/microsoft/FLAML/tree/main/flaml/tune). They can be used beyond the AutoML context. 
+And they can be used in distributed HPO frameworks such as ray tune or nni.
+
 For more technical details, please check our papers.
 
-* [FLAML: A Fast and Lightweight AutoML Library](https://arxiv.org/abs/1911.04706). Chi Wang, Qingyun Wu, Markus Weimer, Erkang Zhu. arXiv:1911.04706, 2020.
-* [Frugal Optimization for Cost-related Hyperparameters](https://arxiv.org/abs/2005.01571). Qingyun Wu, Chi Wang, Silu Huang. To appear in AAAI 2021.
+* [FLAML: A Fast and Lightweight AutoML Library](https://arxiv.org/abs/1911.04706). Chi Wang, Qingyun Wu, Markus Weimer, Erkang Zhu. To appear in MLSys, 2021.
+```
+@inproceedings{wang2021flaml,
+    title={Frugal Optimization for Cost-related Hyperparameters},
+    author={Chi Wang and Qingyun Wu and Markus Weimer and Erkang Zhu},
+    year={2021},
+    booktitle={MLSys},
+}
+```
+* [Frugal Optimization for Cost-related Hyperparameters](https://arxiv.org/abs/2005.01571). Qingyun Wu, Chi Wang, Silu Huang. AAAI 2021.
+* Economical Hyperparameter Optimization With Blended Search Strategy. Chi Wang, Qingyun Wu, Silu Huang, Amin Saied. To appear in ICLR 2021.
 
 ## Contributing
 
@@ -123,9 +142,8 @@ contact [[email protected]](mailto:[email protected]) with any additio
 
 * Chi Wang
 * Qingyun Wu
-* Erkang Zhu
 
-Contributors: Markus Weimer, Silu Huang, Haozhe Zhang, Alex Deng.
+Contributors (alphabetical order): Alex Deng, Silu Huang, John Langford, Amin Saied, Markus Weimer, Haozhe Zhang, Erkang Zhu.
 
 ## License
 

flaml/__init__.py

@@ -1,3 +1,4 @@
+from flaml.searcher import CFO, BlendSearch, FLOW2
 from flaml.automl import AutoML
 from flaml.version import __version__
 import logging