Config param (BC-SECURITY#146)

* add support for --config param. Add documentation on config.yaml for client and server. Add docs for mysql

* formatting

* formatting

README.md

@@ -30,7 +30,7 @@ included author/reference link information in the source of each Empire module a
 properly cite existing or prior work, please let us know at [email protected].
 
 Empire is currently being developed and maintained by [@Cx01N](https://twitter.com/Cx01N_), 
-[@Hubbl3](https://twitter.com/_Hubbl3), & [@Vinnybod](https://twitter.com/_vinnybod). While the main Empire project is 
+[@Hubbl3](https://twitter.com/_Hubbl3), & [@Vinnybod](https://twitter.com/_vinnybod). While the original Empire project is
 no longer maintained, this fork is maintained by [@bcsecurity1](https://twitter.com/BCSecurity1). Please reach out to 
 us on our [Discord](https://discord.gg/P8PZPyf) if you have any questions or want talk about offensive security.
 
@@ -46,7 +46,7 @@ Thank you to the original team of developers: [@harmj0y](https://twitter.com/har
 ## Release Notes
 Please see our [Releases](https://github.com/BC-SECURITY/Empire/releases) or [Changelog](/changelog) page for detailed release notes.
 
-## Empire 4.0 Alpha Documentation
+## Empire 4.0 Documentation
 **Note**: Some things are subject to change before the GA release, and this documentation will (hopefully ;) ) be built 
 out to the wiki before then.
 
@@ -83,7 +83,7 @@ poetry run python empire.py server
 ./ps-empire server -h
 ```
 
-The old embedded client has been removed. To run the new command line client.
+The old embedded client has been removed. To run the new command line client:
 ```sh
 poetry run python empire.py client
 

empire/arguments.py

@@ -14,13 +14,17 @@
 # Client Args
 client_parser.add_argument('-r', '--resource', type=str,
                            help='Run the Empire commands in the specified resource file after startup.')
+client_parser.add_argument('--config', type=str, nargs=1,
+                           help='Specify a config.yaml different from the config.yaml in the empire/client directory.')
 
 # Server Args
 general_group = server_parser.add_argument_group('General Options')
 general_group.add_argument('--debug', nargs='?', const='1',
                            help='Debug level for output (default of 1, 2 for msg display).')
 general_group.add_argument('--reset', action='store_true', help="Resets Empire's database to defaults.")
 general_group.add_argument('-v', '--version', action='store_true', help='Display current Empire version.')
+general_group.add_argument('--config', type=str, nargs=1,
+                           help='Specify a config.yaml different from the config.yaml in the empire/server directory.')
 
 rest_group = server_parser.add_argument_group('RESTful API Options')
 rest_group.add_argument('--restip', nargs=1, help='IP to bind the Empire RESTful API on. Defaults to 0.0.0.0')

empire/client/src/EmpireCliConfig.py

@@ -1,3 +1,4 @@
+import sys
 from typing import Dict
 
 import yaml
@@ -6,12 +7,22 @@
 class EmpireCliConfig(object):
     def __init__(self):
         self.yaml: Dict = {}
-        with open("./empire/client/config.yaml", 'r') as stream:
-            try:
+        if '--config' in sys.argv:
+            location = sys.argv[sys.argv.index('--config') + 1]
+            print(f'Loading config from {location}')
+            self.set_yaml(location)
+        if len(self.yaml.items()) == 0:
+            print('Loading default config.')
+            self.set_yaml("./empire/client/config.yaml")
+
+    def set_yaml(self, location: str):
+        try:
+            with open(location, 'r') as stream:
                 self.yaml = yaml.safe_load(stream)
-            except yaml.YAMLError as exc:
-                print(exc)
-                self.yaml = {}
+        except yaml.YAMLError as exc:
+            print(exc)
+        except FileNotFoundError as exc:
+            print(exc)
 
 
 empire_config = EmpireCliConfig()

empire/server/common/config.py

@@ -1,3 +1,4 @@
+import sys
 from typing import Dict
 
 import yaml
@@ -6,12 +7,22 @@
 class EmpireConfig(object):
     def __init__(self):
         self.yaml: Dict = {}
-        with open("./empire/server/config.yaml", 'r') as stream:
-            try:
+        if '--config' in sys.argv:
+            location = sys.argv[sys.argv.index('--config') + 1]
+            print(f'Loading config from {location}')
+            self.set_yaml(location)
+        if len(self.yaml.items()) == 0:
+            print('Loading default config.')
+            self.set_yaml("./empire/server/config.yaml")
+
+    def set_yaml(self, location: str):
+        try:
+            with open(location, 'r') as stream:
                 self.yaml = yaml.safe_load(stream)
-            except yaml.YAMLError as exc:
-                print(exc)
-                self.yaml = {}
+        except yaml.YAMLError as exc:
+            print(exc)
+        except FileNotFoundError as exc:
+            print(exc)
 
 
 empire_config = EmpireConfig()

empire/server/config.yaml

@@ -18,4 +18,4 @@ database:
     #   format is "192.168.1.1,192.168.1.10-192.168.1.100,10.0.0.0/8"
     ip-blacklist: ""
 modules:
-  retain-last-value: true
+  retain-last-value: false

empire/server/database/base.py

@@ -12,7 +12,6 @@
 
 database_config = empire_config.yaml.get('database', {})
 
-# Note: MySQL not supported yet.
 if database_config.get('type') == 'mysql':
     url = database_config.get('url')
     username = database_config.get('username') or ''

empire/server/database/models.py

@@ -214,7 +214,7 @@ class Profile(Base):
     name = Column(String(255), primary_key=True)
     file_path = Column(String(255))
     category = Column(String(255))
-    data = Column(String, nullable=False)
+    data = Column(Text, nullable=False)
     created_at = Column(UtcDateTime, nullable=False, default=utcnow())
     updated_at = Column(UtcDateTime, default=utcnow(), onupdate=utcnow(), nullable=False)
 

wiki/Configuration.md

@@ -1 +1,123 @@
-_TODO_
+## Configuration
+The Empire Client and Empire Server both have a config.yaml file to customize behavior.
+
+### Server
+The Server configuration is managed via [empire/server/config.yaml](../empire/server/config.yaml).
+- **suppress-self-cert-warning** - Suppress the http warnings when launching an Empire instance that uses a self-signed cert
+
+- **database** - Configure Empire's database. Empire defaults to SQLite and has the ability to run with MySQL.
+
+SQLite - The location of the SQLite db file is configurable.
+```yaml
+database:
+  type: sqlite
+  location: empire/server/data/empire.db
+```
+
+MySQL (Beta) - The url, username, and password are all configurable.
+Everything in Empire should be working with MySQL with the exception of Python agents.
+There is additional work needed in order to support them.
+```yaml
+database:
+  type: mysql
+  url: localhost
+  username:
+  password:
+```
+
+The defaults block defines the properties that are initially loaded into the database when it is first created.
+```yaml
+database:
+  defaults:
+    # staging key will first look at OS environment variables, then here.
+    # If empty, will be prompted (like Empire <3.7).
+    staging-key: RANDOM
+    username: empireadmin
+    password: password123
+    obfuscate: false
+    # Note the escaped backslashes
+    obfuscate-command: "Token\\All\\1"
+    # an IP white list to ONLY accept clients from
+    #   format is "192.168.1.1,192.168.1.10-192.168.1.100,10.0.0.0/8"
+    ip-whitelist: ""
+    # an IP black list to reject accept clients from
+    #   format is "192.168.1.1,192.168.1.10-192.168.1.100,10.0.0.0/8"
+    ip-blacklist: ""
+```
+
+- **modules.retain-last-value** - This tells Empire to retain the last values set for a module.
+In Empire 4.0, the modules objects were converted to be stateless, so when a user executes a module,
+it doesn't impact the values seen or set by another user. Set this to `true` if you want to mimic the old
+behavior.
+
+### Client
+The Client configuration is managed via [empire/client/config.yaml](../empire/client/config.yaml).
+
+- **servers** - The servers block is meant to give the user the ability to set up frequently used Empire servers.
+If a server is listed in this block then when connecting to the server they need only type: `connect -c localhost`.
+This tells the client to use the connection info for the server named localhost from the yaml. In addition, if autoconnect is set to `true`, the client will automatically connect to that server when starting up.
+```yaml
+servers:
+  localhost:
+    host: https://localhost
+    port: 1337
+    socketport: 5000
+    username: empireadmin
+    password: password123
+    autoconnect: true
+```
+- **suppress-self-cert-warning** - Suppress the http warnings when connecting to an Empire instance that uses a self-signed cert
+- **shortcuts** - Shortcuts defined here allow the user to define their own frequently used modules and assign a command to them.
+Let's look at 3 distinct examples. All of which can be found in the default [config.yaml](../empire/client/config.yaml)
+```yaml
+shortcuts:
+  powershell:
+    sherlock:
+      module: powershell/privesc/sherlock
+```
+This first example is the simplest example. It adds a `sherlock` command to the Interact menu for Powershell agents. It does not pass any specific parameters.
+
+```yaml
+shortcuts:
+  powershell:
+    keylog:
+      module: powershell/collection/keylogger
+      params:
+        - name: Sleep
+          value: 1
+```
+This next one is slightly more complex in that we are telling the shortcut to set the *Sleep* parameter to 1.
+Note that if there are any other parameters for this module that we don't define, it will use whatever the default value is.
+
+```yaml
+shortcuts:
+  powershell:
+    bypassuac:
+      module: powershell/privesc/bypassuac_eventvwr
+      params:
+        - name: Listener
+          dynamic: true
+```
+This third one gets a bit more complex. Instead of providing a `value` to the parameter, it is marked as `dynamic`.
+This tells the CLI that it expects the user to send the parameters as part of their command. In other words the user needs to type `bypassuac http1` in order for this to execute.
+The parameters are passed in the order they are defined in config.yaml. There are some convenient autocompletes if the field is named `Listener` or `Agent`.
+
+```yaml
+shortcuts:
+  powershell:
+    whoami:
+      shell: whoami
+```
+The last one is much more simple. Instead of running a module, we run a shell command.
+
+- **resource-file** - A resource file is simply a text file with a list of commands to run in order.
+An example txt is shown below
+```yaml
+resource-file: commands.txt
+
+# commands.txt
+listeners
+uselistener http
+set Port 999
+execute
+```

wiki/Home.md

@@ -31,12 +31,9 @@ On the PowerShell side, Empire implements the ability to run PowerShell agents w
 
 Empire relies heavily on the work from several other projects for its underlying functionality. We have tried to call out a few of those people we've interacted with [heavily here](http://www.powershellempire.com/?page_id=2) and have included author/reference link information in the source of each Empire module as appropriate. If we have failed to properly cite existing or prior work, please let us know at [email protected].
 
-Empire is currently being developed and maintained by [@Cx01N](https://twitter.com/Cx01N_), [@Hubbl3](https://twitter.com/_Hubbl3), & [@Vinnybod](https://twitter.com/AZHalcyon). While the main Empire project is no longer maintained, this fork is maintained by [@bcsecurity1](https://twitter.com/BCSecurity1). 
+Empire is currently being developed and maintained by [@Cx01N](https://twitter.com/Cx01N_), [@Hubbl3](https://twitter.com/_Hubbl3), & [@Vinnybod](https://twitter.com/_Vinnybod). While the main Empire project is no longer maintained, this fork is maintained by [@bcsecurity1](https://twitter.com/BCSecurity1).
 Please reach out to us on our [Discord](https://discord.gg/P8PZPyf) if you have any questions or talk about offensive security.
 
-## Documentation
-Empire maintains a web site version of the documentation at [http://www.powershellempire.com](http://www.powershellempire.com). 
-
 ## Help us Improve!
 
 This documentation was organized and built by the PowerShell Empire development team. It is neither complete nor perfect, so any suggestions, corrections, or additions from the community would be greatly appreciated. Please submit any Wiki changes as [Empire Pull Requests](https://github.com/BC-SECURITY/Empire/pulls) using the [Wiki directory](./wiki).

wiki/Installation.md

@@ -21,7 +21,7 @@ git clone https://github.com/BC-SECURITY/Empire.git
 cd Empire
 sudo ./setup/install.sh
 sudo poetry install
-sudo poetry run python empire --rest -n
+./ps-empire server # or ./ps-empire client
 ```
 
 ## Docker
@@ -45,7 +45,7 @@ All image versions can be found at: https://hub.docker.com/r/bcsecurity/empire/
 * All github tagged releases will be deployed using their version numbers (v3.0.0, v3.1.0, etc)
 
 # Community-Supported Operating Systems
-At this time, we are choosing to only support Kali, Debian, and Ubuntu installations, however we will accept pull requests that fix issues or provide installation scripts specific to other operating systems to this wiki.
+At this time, we are choosing to only support Kali, Debian 10, and Ubuntu 20.04 installations, however we will accept pull requests that fix issues or provide installation scripts specific to other operating systems to this wiki.
 
 <!---
 ## Fedora