Skip to content

Latest commit

 

History

History
2848 lines (2578 loc) · 193 KB

CHANGELOG.md

File metadata and controls

2848 lines (2578 loc) · 193 KB
Raw

CHANGELOG

1.3.4 (May 3, 2017)

LeoStorage

  • #644 [Bug] Fixed "badarg" from watchdog on storage node in error log
    • Fixes for #644 did not include in v1.3.3 because leo_wachdog's version was wrong.

Libraries

Leo Project

Others

1.3.3 (Apr 28, 2017)

LeoStorage

  • #573 [Improve] To be configurable the data synchronization
  • #590 [Improve] API to break a data block in AVS
  • #690 [Improve] Track Slow I/O on leo_compact_fsm_worker
  • #593 [Bug] Fixed that reduce not_found logs
  • #624 [Bug] Fixed that messages in a queue can get lost
  • #645 [Bug][MDC-Replication] read-repair, data-compaction, data-recovery feature may replicate objects more than mdc_replication.num_of_replicas_a_dc
  • #641 [Bug] Fixed that MDC replicated objects are missing HTTP headers after upgrading from 1.3.0 to 1.3.2.1
  • #644 [Bug] Fixed "badarg" from watchdog on storage node in error log
  • #668 [Bug] Fixed that leo_storage can be rebooted when lots of badarg from eleveldb happened
  • #704 [Bug] Fixed that possibility of MQ's data loss
  • #713 [Bug] Fixed that replicator/5 can receive messages generated by gen_server:call/cast
  • #660 [Maintenance] Needs to deliver moving AVS directory after launching

LeoGateway

NFS

  • #327 [Improve] Return file attributes on every operation
  • #598 [Improve] Make the response to fsinfo configurable
  • #613 [Improve] nfs_fh3 is not needed to be true random
  • #614 [Improve] Avoid concating list in leo_nfs_file_handler:list_dir
  • #615 [Improve] large_obj_delete_chunks should keep iterating even some error happened
  • #596 [Bug] Fixed that the total file size can be wrong when writing the same file in parallel
  • #599 [Bug] Fixed that some responses don't comply rfc1813
  • #604 [Bug] Fixed that NFS_DUMMY_FILE4S3DIR should be replaced with one including reserved characters
  • #611 [Bug] Fixed that miss error handling in nfsproc3_rmdir_3
  • #612 [Bug] Fixed that readdir entries can leak when NFS client crash

S3-API

  • #483 [Bug] Fixed that S3 Sync feature does not synchronize directories properly
  • #642 [Bug] Fixed that failed to fetch a bucket info while both managers are down after bucket_prop_sync_interval passed
  • #647 [Bug] Fixed that make find_bucket_by_name fail-safe as #642

LeoManager

  • #695 [Bug] Fixed that degrading the get-bucket command
  • #606 [Improve] Able to create a user with access-key and secret-access-key
  • #607 [Improve] Make a user password more secure

Commons

  • #626 [Bug] Fixed that missing error handlings of leo_ordning_reda
  • #643 [Bug] Fixed that a way to remotely check system_version on nodes with the status command
  • #712 [Bug] Fixed that iterator-handler can be leaked
  • #667 [Improve] Set ERL_CRASH_DUMP_SECONDS to a negative value
  • #671 [Improve][leo_backend_db] Implement status callback for eleveldb
  • #617 [Package] /usr/local/bin/leofs-adm should be removed
  • #638 [Package] Running packaged version as non-privileged user

Libraries

Leo Project

Others

1.3.2 (Jan 30, 2017)

1.3.1 (Dec 28, 2016)

1.3.0 (Aug 9, 2016)

1.2.22 (May 20, 2016)

1.2.21 (March 29, 2016)

1.2.20 (March 23, 2016)

  • Improvements
    • #389 erlang To be able to build LeoFS with Erlang 18
    • #442 leo_storage When removing a large size object, a high-latency issue can be happened
    • #445 leo_backend_db May take too much time when restarting leo_storage
    • #446 leo_object_storage Unnecessary a file:position call when opening a AVS file
    • #447 There are some possibilities that some supervisours don't restart their child as we expect
    • #448 leo_storage,leo_mq Reduce queue messages
    • Improve the LeoStorage's read performance
      • #450 leo_object_storage Improve concurrency
      • #467 leo_object_storage Improve concurrency of read operations
    • leo_manager Implement the command history feature to be able to confirm command histories with it
      • It is stored under log/app directory
## command history file:
$ tree -vhD /path/to/leofs/leofs-1.2.20/leo_manager_0/log/app/
./package/leo_manager_0/log/app/
├── [ 103 Mar 17 16:40]  cmd_history -> /path/to/leofs/leofs-1.2.20/leo_manager_0/log/app/cmd_history.20160317.16.1
├── [ 230 Mar 17 16:41]  cmd_history.20160317.16.1
├── [  97 Mar 17 16:40]  error -> /path/to/leofs/leofs-1.2.20/leo_manager_0/log/app/error.20160317.16.1
├── [ 256 Mar 17 16:41]  error.20160317.16.1
├── [  96 Mar 17 16:40]  info -> /path/to/leofs/leofs-1.2.20/leo_manager_0/log/app/info.20160317.16.1
└── [1.6K Mar 17 16:40]  info.20160317.16.1

## command history format:
## | <datetime> | <unixtime> | <command> |
$ tail -f /path/to/leofs/leofs-1.2.20/leo_manager_0/log/app/cmd_history
2016-03-17 16:40:42.685451 +0900    1452670842685495    start
2016-03-17 16:41:22.732225 +0900    1452670882732267    update-log-level [email protected] debug
2016-03-17 16:41:36.745973 +0900    1452670896746008    update-consistency-level 1 1 1

1.2.18 (January 7, 2016)

1.2.16 (Octorber 9, 2015)

1.2.14 (September 11, 2015)

1.2.12 (July 10, 2015)

1.2.11 (June 30, 2015)

1.2.10 (May 22, 2015)

1.2.9 (May 15, 2015)

1.2.8 (Apr 14, 2015)

1.2.7 (Mar 6, 2015)

1.2.6 (Feb 20, 2015)

1.2.5 (Jan 29, 2015)

1.2.4 (Jan 23, 2015)

1.2.3 (Jan 20, 2015)

1.2.2 (Dec 17, 2014)

  • New Features
    • #117 leo_storage Implemented the auto-compaction mechanism
      • The auto-compaction configuration file for LeoFS Storage, name of which is leo_storage.conf
      • The auto-compaction configuration is as follows:
      ## [compaction] enabled compaction? - default:false
autonomic_op.compaction.is_enabled = true

## [compaction] number of parallel procs - default:1
autonomic_op.compaction.parallel_procs = 1

## [compaction] warning ratio of active size - default:70%
autonomic_op.compaction.warn_active_size_ratio = 70

## [compaction] threshold ratio of active size - default:60%
autonomic_op.compaction.threshold_active_size_ratio = 60
  • Improvements
    • #270 Supported leo_watchdog for FreeBSD and SmartOS
    • #281 leo_gateway Made timeout configurable for the get and the find_by_parent_dir
    • The watchdog mechnism to production status
      • The watchdog configuration file for LeoFS Storage and LeoFS Gateway, name of which are leo_storage.conf and leo_gateway.conf
      • The watchdog configuration for CPU and Disk as follows:
      ##  Watchdog.CPU
##
## Is cpu-watchdog enabled - default:false
watchdog.cpu.is_enabled = true

## cpu - raised error times
watchdog.cpu.raised_error_times = 3

## cpu - watch interval - default:5sec
watchdog.cpu.interval = 5

## Threshold CPU load avg for 1min/5min - default:5.0
watchdog.cpu.threshold_cpu_load_avg = 5.0

## Threshold CPU load util - default:100 = "100%"
watchdog.cpu.threshold_cpu_util = 100

##  Watchdog.DISK
##
## Is disk-watchdog enabled - default:false
watchdog.disk.is_enabled = true

## disk - raised error times
watchdog.disk.raised_error_times = 3

## disk - watch interval - default:1sec
watchdog.disk.interval = 1

## Threshold disk use% - defalut:85%
watchdog.disk.threshold_disk_use = 85

## Threshold disk util% - defalut:100%
watchdog.disk.threshold_disk_util = 100

## Threshold disk read kb/sec - defalut:262144(KB)
watchdog.disk.threshold_disk_rkb = 262144

## Threshold disk write kb/sec - defalut:262144(KB)
watchdog.disk.threshold_disk_wkb = 262144

## disk target devices for checking disk utilization
watchdog.disk.target_devices = []
      • Improved MQ's confurations for LeoFS Stroage in order to be able to control system load
      ## [Number of bach processes of message]
##
## Minimum number of bach processes of message
mq.num_of_batch_process_min = 100

## Maxmim  number of bach processes of message
mq.num_of_batch_process_max = 10000

## Maxmim  number of bach processes of message
mq.num_of_batch_process_regular = 1000

## Maxmim  number of bach processes of message
mq.num_of_batch_process_step = 250

## [Interval beween batch-procs]
##
## Minimum value of interval beween batch-procs(msec)
mq.interval_between_batch_procs_min = 10

## Maximum value of interval between batch-procs(msec)
mq.interval_between_batch_procs_max = 1000

## Regular value of interval between batch-procs(msec)
mq.interval_between_batch_procs_regular = 300

## Step value of interval between batch-procs(msec)
mq.interval_between_batch_procs_step = 100
      • Improved the mq-status command of leofs-adm
      $ leofs-adm mq-stats [email protected]
              id                |    state    | number of msgs | batch of msgs  |    interval    |            description
--------------------------------+-------------+----------------|----------------|----------------|-----------------------------------
leo_delete_dir_queue            |   idling    | 0              | 1000           | 100            | delete directories
leo_comp_meta_with_dc_queue     |   idling    | 0              | 1000           | 100            | compare metadata w/remote-node
leo_sync_obj_with_dc_queue      |   idling    | 0              | 1000           | 100            | sync objs w/remote-node
leo_recovery_node_queue         |   idling    | 0              | 1000           | 100            | recovery objs of node
leo_async_deletion_queue        |   idling    | 0              | 1000           | 100            | async deletion of objs
leo_rebalance_queue             |   running   | 2167           | 1400           | 10             | rebalance objs
leo_sync_by_vnode_id_queue      |   idling    | 0              | 1000           | 100            | sync objs by vnode-id
leo_per_object_queue            |   idling    | 0              | 1000           | 100            | recover inconsistent objs
  • Fixed Bugs
    • #273 leo_gateway NFS Max file size was hardcoded as 4GB
    • #274 leo_manager leo_storage leo_gateway Crashed a manager node by snmp-related issue with Erlang-17
    • #275 leo_gateawy An error message was output if there is no the http custom header file
    • #277 leo_gateway In REST mode, Putting a large object failed
    • #279 leo_object_storage Compaction status can be inconsistent when an error occured while preparing compaction
    • #282 leo_gateway Gateway's timeout configuration has been ignored
  • Used libraries

1.2.1 (Nov 11, 2014)

1.2.0 (Nov 7, 2014)

1.1.5 (Oct 15, 2014)

1.1.4 (Oct 3, 2014)

1.1.3 (Sep 12, 2014)

1.1.2 (Sep 4, 2014)

1.1.1 (Aug 21, 2014)

1.1.0 (Aug 6, 2014)

1.0.2 (May 9, 2014)

1.0.1 (May 8, 2014)

1.0.0 (Apr 2, 2014)

1.0.0-pre3 (Feb 20, 2014)

1.0.0-pre2 (Jan 22, 2014)

1.0.0-pre1 (Dec 27, 2013)

0.16.8 (Dec 12, 2013)

0.16.5 (Nov 26, 2013)

0.16.0 (Oct 25, 2013)

0.14.9 (Sep 27, 2013)

0.14.8 (Sep 12, 2013)

0.14.7 (Aug 23, 2013)

0.14.6 (July 11, 2013)

0.14.5 (July 8, 2013)

0.14.4 (June 28, 2013)

0.14.3 (June 21, 2013)

0.14.2 (June 7, 2013)

0.14.1-1 (May 8, 2013)

  • Improved
    • Gateway
      • Upgrade Ranch and Cowboy to latest version
  • Fix Bugs
    • Gateway
      • Reply empty response unintentionally when using some s3 clients
        • "leo_gateway" stored a "0-byte length object" into the cache when put an large-object(over 5MB) from clients. So We supported to remove a registered object from the cache.

0.14.1 (Apr 15, 2013)

0.14.0 (Mar 20, 2013)

0.14.0-RC2 (Mar 14, 2013)

0.14.0-RC1 (Mar 5, 2013)

0.12.7 (Dec 26, 2012)

0.12.5 (Dec 7, 2012)

0.12.4 (Nov 21, 2012)

0.12.3 (Nov 9, 2012)

0.12.2 (Nov 2, 2012)

0.12.1 (Oct 25, 2012)

0.12.0 (Oct 20, 2012)

0.10.2 (Sep 25, 2012)

  • Improve performances
    • NOT used "proplists:get_value" function
      • Replace from "proplists:get_value/2,3" to "lists:keyfind/2"
    • Related libs:
      • leo_gateway
      • leo_storage
      • leo_manager
      • leo_logger
      • leo_mq
      • leo_object_storage
      • leo_ordning_reda
      • leo_redundant_manager
      • leo_s3_libs
      • leo_statistics
  • Improve leo_manager
    • Support output of json-format
    • Support multi-ports TCP-server
      • for CUI console
      • for Application (JSON-Format)
  • Improve leo_storage
    • Support plural devices
    • Reduced rebalance/recover costs
      • Support compression of stacked objects and decompresson of received objects (Using snappy)
  • Fix bugs
    • leo-gateway related (S3-related):
      • Create bucket from 'Dragon Disk'
      • When using cowboy can send bad values(not iodata)

0.10.1 (Sep 12, 2012)

  • Improve leo_hex performances
    • "binary_to_hex" function
    • Related libs:
      • leo_gateway
      • leo_object_storage
      • leo_redundant_manager
    • By this correspondence, LeoFS's performance improved 20% up.
  • Improve leo_manager
    • Format of output from manager-console
      • Commands:
        • "status"
        • "whereis"
  • Improve performance of "storage-stats" in manager-console
  • Fix bugs
    • A handling error in S3-libs

0.10.0 (Aug 30, 2012)

  • Improve S3-API's compatibility
    • Add S3-authentication
    • Add S3-bucket
  • Add S3-related command in LeoFS's manager
    • "s3-gen-key" : Generate a S3 key pair(AccessKeyID and SecretAccessKey)
    • "s3-set-endpoint" : Register a new S3 Endpoint
    • "s3-delete-endpoint" : Delete a S3 Endpoint
    • "s3-get-endpoints" : Retrieve all of S3 Endpoints registered
    • "s3-get-buckets" : Retrieve all of Buckets registered
  • Improve order of system launch
    • Remove "attach command" in manager - After Storage launched, the node's state is automatically changed to "attached"
  • Improve rebalance-function's performance which is about 5 times compare with v0.9.1
  • Improve compact-function. Restrain storage's load when compact objects.
  • Fix bugs
    • Deletion of Zero bytes in Storage
    • Behavior after the restart of Manager
    • Re-register procs into the Manager's monitor

0.9.1 (Jul 13, 2012)

  • Remove "apps" directory in leofs - Modified "reltool.config"
  • Fix 'shadow vars'
  • Remove ZMQ log-appender and AMQP log-appender for leo_logger, They will be provided LeoFS's sub projects.
  • Improve - In order to be able to extend "LeoFS's Object Container's file format".

0.9.0 (Jul 4, 2012)

  • Initial release