Changelog

v1.5.0

Summary:

changes and optimizations for making NEF_emulator capable of running bigger scenarios

UE movement approach change:

old: iterate over all path-points and simulate speed by using sleep() (LOW=1sec HIGH=0.1sec)

new: constantly use sleep(1sec) and simulate speed by skipping (or not) path-points

more on the pros/cons of this approach can be found at the relative source code section, the old one is commented out

update of leaflet.js to version 1.8.0 (we've indetified a bug when closing mark tooltips, it's supposed to be fixed by the maintainers of the project at the upcoming release)

UI changes

dashboard-cells.js minor fix to display error details correctly in the toast message
🪛 fix /map js console errors caused by the UEs layer-checkbox (access to null marker)
🪛 fix /map UEs buttons: handle case of UEs with no paths assigned
/dashboard page change: instead of 2 consecutive API requests on UE Save 👇:
- 1 API request to assign path everytime the user selects something different
- 1 API request on Save button
/map page: add type-of-service column to datatable (cells now display Monitoring Event API or AsSession With QoS API)
/login: add "hit enter --> submit form" functionality
/register: add "hit enter --> submit form" functionality
add NEF logo
move part of login.js code to app.js (more clean approach + added app.default_redirect variable)
maps.js: increase timeouts to 60 sec (edge case with >200 UEs, start/stop takes time)
maps.js: add api_get_moving_UEs() to only retrieve moving UEs ➡ move part of ui_map_paint_UEs() to ui_map_paint_moving_UEs()
app.js: move api_test_token() outside document.ready() for quicker user auth checks
401 page redirect: when the token can't be validated the user is redirected to a 401 Unauthorized page and after a few seconds is redirected to /login. Previously, the user was redirected to login without being notified.
map.js: optimize helper_check_path_is_already_painted( path_id ) by replacing the simple array of painted paths with a key-value object

Backend

⛔ for optimization purposes, the UEs movement is handled in memory (no more intensive read/writes to Postgres) 👇
➕ api/v1/ue_movement/state-ues now returns moving UEs information only. It helps with the edge cases of having many UEs and only a few of them actually moving around
create new module/file for threads utils.py ➡ ue_movement.py
- ⛔ /utils/state-loop/{{supi}} ➡ /ue_movement/state-loop/{{supi}}
- ⛔ /utils/start-loop ➡ /ue_movement/start-loop
- ⛔ /utils/stop-loop ➡ /ue_movement/stop-loop
utils.py: add a 2nd approach for making the UEs move within their path and control their speed (see #2eb19f8)
SQLAlchemy: add pool_size=150, max_overflow=20 to create_engine( ... )
fix NoneType exception on MonitoringEvent one time request when cell is None
Add middleware to return custom response header X-Process-Time that counts request-response proccesing time
Split callbacks in two files 👉 From send_callback.py ➡ monitoring_callbacks.py + qos_callback.py
fix callback notification for QoS after the transition from db to memory

Database

postgreSQL add command: -c shared_buffers=256MB -c max_connections=200 to docker-compose
MonitoringEvent: migration from postgreSQL to MongoDB 👇
- fix check_numberOfReports function accordingly

Libraries

upgrade leaflet.js (1.7.1 to 1.8.0)

v1.4.1

Migration to Python 3.9

(see commit: 2ef55ed)

updated tiangolo/uvicorn -gunicorn -fastapi:python3.9 (from 3.7)
updated python = "3.9.7" (from 3.7)
updated uvicorn = "^0.17.6" (from 0.15.0)
updated fastapi = "^0.78.0" (from 0.54.1)
updated pymongo = "^4.1.0" (from 3.12.1)
updated requests = "^2.27.0" (from 2.23.0)
updated gunicorn = "^20.1.0" (from 20.0.4)
removed celery = "^4.4.2"
removed alembic = "^1.4.2"
removed GeoAlchemy2 = "^0.9.4"

📄 docs

add documentation for the UI (historical context, naming conventions, front-end libraries & approaches used etc...)

Backend

🚫🤚 Forbid user to update cells while UEs are moving
Add timeout values in requests.request 👇
- Timeout values according to https://docs.pythonrequests.org/en/master/user/advanced/#timeouts
- Fixes "hanging" UEs problem: when subscribing to NEF APIs, specific IPs included in the notificationDestination URL caused timeouts. Since the default for Python Requests is None (wait until the connection is closed) the UE "freezes" until the request times out and the exception is finally caught.

v1.4.0

Scenario `import` / `export`

⛔ breaking change: json data of exported scenarios from v1.3.x will not able to be imported to v1.4.0. Users will have to recreate them manually.

UI changes

/dashboard js split (see issue: #35):
- dashboard.js keeps only the document.ready function + some helper ones
- dashboard-gnbs.js has the gNBs part
- dashboard-cells.js has the Cells part
- dashboard-ues.js has the UEs part
- dashboard-paths.js has the Paths part
/dashboard split goals:
- ✅ use beforeSend, complete and spinners for making visible that reload takes place in the background. Locally this happens so fast that the user doesn't notice.
- ✅ change api_get/put/delete/post() functions to return to callbacks for UI actions
- ✅ introduce ui_fetch_and_update() functions that use the above API calls and after they fetch data, they update the UI. This helped us prevent some states with "stale" data
- ✅ add layers to maps to let the users choose what they want to see or not (Cells, UEs, Paths) when adding / editing
/dashboard fixes & improvements
- increase map 🗺️ height inside every modal from 300 to 600px
- add_cell modal: fix NaN errors when the user doesn't click on the map to generate X,Y for the new cell
- ui_map_paint_path() function can now take opacity argument
- 🪛 handle cell_id_hex == null: display - in Datatable cells
/map:
- 🪛 map.js minor fix: add a list of painted paths to first check and -if not already added on the map- continue painting it. (The problem appeared visually when multiple UEs had the same path assigned)
- ✅ handle cell_id_hex == null and display grey ⚪ UE-icon when not connected to any cell (this also applies to every UE just after an import)

NEF APIs / backend

Fix problem with path_id on import/export scenario. (e.g., if we export a scenario with UEs with path_ids 1,2,4, when the scenario is imported, the path with id 4 is added as path with 3. The UE is successfully correlated with the new path id 3)
🙅‍♂️Forbid user to update (gnb/cell)'s hex ids, if they already exist
Add disconnected state functionality. 👉 When there is no radio coverage 📵 the UE disconnects from the cell that it's currently connected.

Docker 🐳

🔥 hotfix make build / make build-no-cache: add --profile option to docker-compose

Libraries

🪛 Fix build error caused by jinja2 newer version (lock to v.3.0.3)

Other

✔ make db-reset : except for reseting the postgresql db, add functionality to also reset mongo db

v1.3.2

Fix UE-association-path selection with path_id 0 (no path selected) - both dashboard and backend
Fix bug in check_expiration_time function
The Monitoring Event subscription by external id is retrieved by additional filter owner_id

v1.3.1

Fix endpoints on MonitoringEvent API {apiroot}/nef/api/v1/3gpp-monitoring-event/v1/{scsAsId}/subscriptions

v1.3.0

License ⚖️

added Apache-2.0 License

Docker 🐳

⛔ breaking change, 🔃 upgrade docker-compose to version 1.29.2, build 5becea4c

Guidelines: head over to docs.docker.com/compose/install/ and choose your OS. For Linux you can use the following 👇:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Reason: our docker-compose.yml now uses profiles (docs.docker.com/compose/profiles/). This gives us the option of adding some kind of labeling to every container, for example ["dev", "debug", "frontend"] that can later be passed to docker-compose up. Practically this means that you spin up the containers selectively depending on what you need. With the relative changes to the Makefile:
- ℹ️ make up now runs only the 3 containers needed for the NEF_emulator
- ✔ make debug-up: new command, runs the full composition: 5 containers, 3 needed for NEF + 2 for Database management (Postgres & MongoDB)
```
make up:
       docker-compose --profile dev up

make debug-up:
       docker-compose --profile debug up
```

NEF APIs / backend

split documentation in two parts:
- NEF_emulator (Swagger - /docs | Redoc /redoc)
- Northbound APIs (Swagger - /nef/docs | Redoc /nef/redoc)
⛔ breaking change, split endpoints in two parts:
- NEF emulator {apiroot}/api/v1/{NEF_emulator endpoints}
- Northbound APIs {apiroot}/nef/api/v1/{Northbound endpoints}
✔ added functionality to export/import the scenario (gNBs, Cells , UEs, paths) (see UI changes)
- /api/v1/utils/export/scenario
- /api/v1/utils/import/scenario

UI changes

🔃 upgrade coreui to v4.1.0 which fixes a persistent browser console error
✔ /export page to generate json with the scenario (gNBs, Cells, UEs, paths)
✔ /import page to upload json and create a previously exported scenario
📄 /docs: added sidebar links to the new API Swagger/ReDoc pages (after the UI / North Bound split)
minor changes to the UE datatable to show more handy attributes (e.g. external_identifier, path...)
✔ added 2 Javascript libraries: codemirror & highlight for json highlighting inside textareas
/dashboard: added some callback functions needed at api_put_UE_callback(...) & api_post_UE_callback(...)
/dashboard: added handling for UEs without selected path

Testing

Initial definition of functional tests for MonitoringEvent API and AsSessionWithQoS API under docs/test_plan

v.1.2.0

Docker 🐳

⚠ new containers added to the composition
- mongo:4.4.10
- mongo-express:1.0.0-alpha.4

this practically means that you will have to: 👇👇👇

make down-v           #remove the old containers & volumes
make build            #build the new backend (pymongo was added)
make dev-prepare-env  #use the new .env file
make up               #start the services
make db-init          #add data (optional)

NEF APIs / backend

new "Session With QoS" endpoints
- GET /api/v1/3gpp-as-session-with-qos/v1/{scsAsId}/subscriptions ✔ added
- POST /api/v1/3gpp-as-session-with-qos/v1/{scsAsId}/subscriptions ✔ added
- GET /api/v1/3gpp-as-session-with-qos/v1/{scsAsId}/subscriptions/{subscriptionId} ✔ added
- PUT /api/v1/3gpp-as-session-with-qos/v1/{scsAsId}/subscriptions/{subscriptionId} ✔ added
- DELETE /api/v1/3gpp-as-session-with-qos/v1/{scsAsId}/subscriptions/{subscriptionId} ✔ added
Callback notification functionality for "Session With QoS"
- The event that triggers the notification is based on handover
new "QoS Information" endpoints
- GET /api/v1/qosInfo/qosCharacteristics ✔ added This endpoint returns some standardized 5QIs that are loaded from the app/core/config/qosCharacteristics.json file
- GET /api/v1/qosInfo/qosProfiles/{gNB_id} ✔ added This endpoint returns the QoS Profiles that have been created and sent to the gNB when a user makes a subscription with QoS for a UE.
Monitoring Event API
- Addition of ipv4 both in callback notification and in 201 Created Response
- Fix swagger documentation after ipv4 change in callbacks
- Forbid duplicate subscriptions for the same external id
UEs
- Fix schemas
- Change ipv6 address format. Now the ipv6 is stored in exploded mode (e.g., 0000:0000:0000:0000:0000:0000:0000:0001)
- Forbid user to delete UE, while it's moving
- Forbid user to change UE's path (while UE is moving)
- Validate UE's ids on create/update. (i.e., supi, ipv4, ipv6, mac address, external id)
- Initiate UE's movement from a random point
backend server port (:8888) is now configurable via the .env file
host.docker.internal now reachable from inside the container, to allow callbacks to services running directly on the host
Fix cell/gNB ids at the following endpoints (i.e., from database ids to actual cell/gNB ids)
- /api/v1/Cells/by_gNB/{gNB_id}
- /api/v1/UEs/by_gNB/{gNB_id}
- /api/v1/UEs/by_Cells/{cell_id}
/frontend/location/ ⛔ deprecated and replaced by /path
⚠ endpoint trailing slashes / trigger a 307 redirect (prefer /path not /path/)

UI changes

/map add search/filter option to datatables
the users can now generate their own scenarios on the map 🗺 more easily:
- /dashboard add CRUD operation buttons for gNBs, Cells, UEs, Paths
- /dashboard add CRUD operation modal windows for gNBs, Cells, UEs, Paths
- color attribute is now used when paths are displayed on the map
- /dashboard add toastr js to display messages

Other

⛔ make db-init-simple is deprecated and replaced by make db-init
✔ make db-reinit can now be used as a shortcut of: make db-reset -> make db-init
📄 docs: guidelines added on how to git switch to specific tag
📄 docs: different network architectures added for NEF <--> NetApp communication options
code cleanup + comments

Libraries

added pymongo = "^3.12.1"

v.1.1.0

NEF APIs / backend

endpoints summary:
- GET /api/v1/utils/monitoring/notifications ✔ added
- GET /api/v1/utils/monitoring/last_notifications ✔ added
- POST /api/v1/3gpp-monitoring-event/v1/{scsAsId}/subscriptions ⛔ breaking change
⛔ The subscription for the Monitoring Event API is now achieved through the external Id (e.g., 10001@domain.com). The ipv4, ipv6, msisdn ids have been removed from the Monitoring Event API. Moreover, external id and the reference resource subscription are included in callback requests.
✔ the 2 newly added endpoints are to be used by the UI for fetching and displaying subsctription events. How they work:
- Every event has a unique number / ID
- The backend keeps on-the-fly a dictionary with the 100 latest events. This way the UI is always able (after a page reload) to show the latest 100 events if they exist
- On "page load/reload" the UI asks the above list of events
- On "polling for new events" the UI provides the number / ID of the latest event that has already received and the backend sends back the new events that may have been occurred. For example, the UI provides that it has received up to event 154 and the backend sends back events 155, 156 and 157 which have taken place in the meanwhile (time between two polling requests)

UI changes

/map: minor changes to markers (UE tooltips, Cell details, more info etc...)
/map: add "fitbounds functionality" to configure the center of the map depending on the cells loaded for the scenario
/map: add datatables to show subscription events (fetched from the newly added endpoints)
/map: add select buttons with different interval options, to let the user choose how frequently the data are updated (UEs position & subscription events)
/map: add modal that shows more details from a subscription event (request body, response body etc...)

Other

Makefile: add logs-location command to filter logs
add comments to js source code
override ./start-reload.sh to prevent uvicorn reloads being triggered from changes to html, css and js files: this affects @tiangolo's docker image default behavior. For more details check the backend Dockerfile

Libraries

upgrade uvicorn to version 0.15.0

Files

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

Changelog

v1.5.0

UI changes

Backend

Database

Libraries

v1.4.1

Migration to Python 3.9

📄 docs

Backend

v1.4.0

Scenario import / export

UI changes

NEF APIs / backend

Docker 🐳

Libraries

Other

v1.3.2

v1.3.1

v1.3.0

License ⚖️

Docker 🐳

NEF APIs / backend

UI changes

Testing

v.1.2.0

Docker 🐳

NEF APIs / backend

UI changes

Other

Libraries

v.1.1.0

NEF APIs / backend

UI changes

Other

Libraries

Scenario `import` / `export`