Skip to content

Releases: hopshadoop/hops

2.8.2.3

24 May 09:26
Compare
Choose a tag to compare
    Release notes - hops - Version 2.8.2.3

Bug

  • [HOPS-109] - Quota management of Small Files
  • [HOPS-136] - Container logging does handle lost containers
  • [HOPS-137] - Quota is not recorded appropriately for small files which result in negative quota when deleting files/directories
  • [HOPS-140] - garbage container logs and checkpoint remaining int the database
  • [HOPS-143] - Quota multiplicator fixed period does not behave according to specification
  • [HOPS-150] - Test code was commited by accident and block from having more than one gpu
  • [HOPS-152] - app get stuck in NEW_SAVING when user name too long
  • [HOPS-155] - Store small files on the datanodes if the database disk tables are full.
  • [HOPS-157] - Nullpointer and Index out of bound since HOPS-152
  • [HOPS-170] - Slow deletionof very large directories
  • [HOPS-185] - Containers requiring GPUs should not be able to reserve resources on NodeManagers with no GPUs.
  • [HOPS-322] - interferences between TransactionalRequestHandler and LightWeightRequestHandler
  • [HOPS-337] - replication Index stuck in UnderReplicatedBlocks
  • [HOPS-365] - NullPointerException when trying to role back transaction
  • [HOPS-366] - Datanode not doing block report after exception
  • [HOPS-368] - NullPointerException when trying to invalidate blocks on dead datanode
  • [HOPS-396] - Maximum GPU allocation gets set to 0

New Feature

  • [HOPS-141] - Monitor and control CertificateLocalizationService via JMX

Improvement

  • [HOPS-131] - Base price should be different for GPU resource and for General resources
  • [HOPS-133] - Remove deploy-server.sh from hops-metadata-dal-impl-ndb and hops-gpu-management-impl-nvidia
  • [HOPS-138] - Increase detail of log output for processed block report
  • [HOPS-139] - Add another method for ContentSummary to return the results directly from inodes_attributes in quote enabled directories
  • [HOPS-144] - Disabled uber pricing
  • [HOPS-149] - Add app IDs in project daily cost
  • [HOPS-151] - App's GPU usage in Yarn UI
  • [HOPS-158] - Refactor HopsSSLSocketFactory and add check for Environment Variable set with crypto material
  • [HOPS-164] - fix hops schema updates for Flyway
  • [HOPS-166] - change flyway storage engine from innodb to ndb
  • [HOPS-350] - Update ClusterJ and libndbclient to 7.5.10

2.8.2.3.rc0

31 Jan 12:21
Compare
Choose a tag to compare
2.8.2.3.rc0 Pre-release
Pre-release
    Release notes - hops - Version 2.8.2.3

Bug

  • [HOPS-109] - Quota management of Small Files
  • [HOPS-136] - Container logging does handle lost containers
  • [HOPS-137] - Quota is not recorded appropriately for small files which result in negative quota when deleting files/directories
  • [HOPS-140] - garbage container logs and checkpoint remaining int the database
  • [HOPS-143] - Quota multiplicator fixed period does not behave according to specification
  • [HOPS-150] - Test code was commited by accident and block from having more than one gpu
  • [HOPS-152] - app get stuck in NEW_SAVING when user name too long
  • [HOPS-155] - Store small files on the datanodes if the database disk tables are full.
  • [HOPS-157] - Nullpointer and Index out of bound since HOPS-152

New Feature

  • [HOPS-141] - Monitor and control CertificateLocalizationService via JMX
  • [HOPS-147] - Reloadable key store manager

Improvement

  • [HOPS-131] - Base price should be different for GPU resource and for General resources
  • [HOPS-133] - Remove deploy-server.sh from hops-metadata-dal-impl-ndb and hops-gpu-management-impl-nvidia
  • [HOPS-138] - Increase detail of log output for processed block report
  • [HOPS-139] - Add another method for ContentSummary to return the results directly from inodes_attributes in quote enabled directories
  • [HOPS-144] - Disabled uber pricing
  • [HOPS-149] - Add app IDs in project daily cost
  • [HOPS-151] - App's GPU usage in Yarn UI
  • [HOPS-153] - [hops-hadoop-chef] Secure webhdfs REST API
  • [HOPS-158] - Refactor HopsSSLSocketFactory and add check for Environment Variable set with crypto material

2.8.2.2

31 Jan 12:23
Compare
Choose a tag to compare
    Release notes - hops - Version 2.8.2.2

Bug

  • [HOPS-95] - Differentiate certificate localization directory for RM and NMs
  • [HOPS-96] - Import HDFS-6570 to fix checkAccess issues with Hive
  • [HOPS-99] - Add makeQualified method to HdfsFileStatus and HdfsLocatedFileStatus
  • [HOPS-100] - Fix ssl-server.xml path in HopsSSLSocketFactory
  • [HOPS-102] - Erasure Coding Tests are failing because of excessive memory use
  • [HOPS-104] - Missing check-refresh causes class-cast exception on block report of UC blocks.
  • [HOPS-120] - TestErasureCodingManager regularly run out of memory
  • [HOPS-123] - Guard for no data in phantom blocks
  • [HOPS-127] - pom <repositories> and <distributionManagement> badly configured
  • [HOPS-128] - libhopsyarn pointing to wrong table
  • [HOPS-129] - The update schema script from 2.8.2.1 to 2.8.2.2 should update yarn_price_multiplicator
  • [HOPS-130] - Container log remaining in DB for containers stoped just after creation
  • [HOPS-136] - Container logging does handle lost containers
  • [HOPS-140] - garbage container logs and checkpoint remaining int the database
  • [HOPS-150] - Test code was commited by accident and block from having more than one gpu

New Feature

  • [HOPS-98] - Quota for GPU hours
  • [HOPS-124] - libhopsyarn and libhopsnvml shouldn't be included in the pom files.

Improvement

  • [HOPS-94] - Improve error handling for HopsSSLSocketFactory
  • [HOPS-97] - Use logicaltime per inode to order logs on the same file
  • [HOPS-105] - Parallellize add-stored block after calculating report diff to improve reporting performance
  • [HOPS-119] - Add certificate password as container local resource
  • [HOPS-122] - Recovering ssl certificate info
  • [HOPS-125] - Cleanup hops git repo from hs_err*.log files
  • [HOPS-138] - Increase detail of log output for processed block report

2.8.2.1

31 Jan 12:22
Compare
Choose a tag to compare
    Release notes - hops - Version 2.8.2.1

Bug

  • [HOPS-14] - remove thread in NamenodeSelector
  • [HOPS-16] - Transactions deadlock exception is thrown when hundreds of clients simultaneously read the same file.
  • [HOPS-20] - Lease manager dies on Transient exception.
  • [HOPS-21] - Poor Synchronous implementation of Namenode Selector in HopsFS 2.7.3 causes 80-90% drop in throughput
  • [HOPS-45] - Slow LS operation on directories with large number of immediate descendants
  • [HOPS-47] - TestUsersGroups fail due to wrong order of list elements
  • [HOPS-48] - FATAL UniqueKeyConstraintViolationException During Namenode Creation
  • [HOPS-51] - hdfs native_tests failing
  • [HOPS-56] - setOwner fails with ForeignKeyConstraintViolationException due to outdated cache
  • [HOPS-59] - HopsSSLSocketFactory assumes there is a single proxy superuser
  • [HOPS-67] - Incoming data are not processed in Server
  • [HOPS-69] - Delete directoy does not work non leader namenode when quota is enabled
  • [HOPS-71] - Fix permission issue when writing to Cgroup device subsystem
  • [HOPS-75] - Fix Log Aggregation in Yarn
  • [HOPS-76] - ProcfsBasedProcessTree cannot parse PID process tree info
  • [HOPS-81] - Missing version specifier in hadoop-common dependency causes warnings
  • [HOPS-82] - Hops-GPU should handle cases when a subset of GPUs are not working correctly
  • [HOPS-84] - hadoop-common pom.xml must include hops-gpu-management as dependency
  • [HOPS-85] - Change collate for all the remaining yarn_* tables
  • [HOPS-89] - Owner not correctly set in Cgroup hierarchy
  • [HOPS-90] - Cgroups broken due to bug in container-executor
  • [HOPS-91] - DFSClient update NN list as superuser
  • [HOPS-92] - Symbol lookup error crashes NodeManager
  • [HOPS-93] - Refactor hops-gpu-manage and hops-gpu-management-impl-nvidia
  • [HOPS-103] - Concurrency bug in blockreporting causes iterator infinite loop
  • [HOPS-107] - Small file's read fails when a decommissioned node is selected as a phanton datanode
  • [HOPS-108] - Get HDFS username from environment variable in libhdfs
  • [HOPS-110] - Change column type of small files tables from varchar to varbinary
  • [HOPS-111] - Support set replication api for small files
  • [HOPS-112] - Implement seek api for small files
  • [HOPS-114] - Can't build without -Pndb since [HOPS-103]
  • [HOPS-116] - Spark applications fail when small fiels are read using read(ByteBuffer) api
  • [HOPS-123] - Guard for no data in phantom blocks
  • [HOPS-127] - pom <repositories> and <distributionManagement> badly configured
  • [HOPS-128] - libhopsyarn pointing to wrong table
  • [HOPS-130] - Container log remaining in DB for containers stoped just after creation
  • [HOPS-136] - Container logging does handle lost containers
  • [HOPS-140] - garbage container logs and checkpoint remaining int the database
  • [HOPS-150] - Test code was commited by accident and block from having more than one gpu

Sub-task

  • [HOPS-41] - NPE in Rename for metaEnabled dirs and subdirs
  • [HOPS-42] - Optimize logging of rename in the same meta enabled directory
  • [HOPS-43] - Delete after rename of a metaEnabled directory fail
  • [HOPS-55] - Refactoring block report processing
  • [HOPS-58] - Correcting code quality issues in FSNamesystem
  • [HOPS-60] - Remove remains of Federation from HopsFS
  • [HOPS-62] - Replace Block Report RPC with Bucketized RPC
  • [HOPS-63] - Create a HashBucket entity
  • [HOPS-64] - Replace the block reporting mechanism with the hash-based one

New Feature

  • [HOPS-2] - Improve Small File Performance in HopsFS
  • [HOPS-4] - Uber style resource price
  • [HOPS-7] - TLS support for Hadoop RPC/IPC
  • [HOPS-11] - GPU support for HopsYARN (Umbrella)
  • [HOPS-12] - Create an abstraction layer for GPU devices
  • [HOPS-13] - Create an implementation layer for GPU devices
  • [HOPS-19] - Truncate support in HDFS
  • [HOPS-23] - avoid running app to be able to access other app data
  • [HOPS-124] - libhopsyarn and libhopsnvml shouldn't be included in the pom files.

Improvement

  • [HOPS-17] - Optimizing addBlock and incrementalBlockReport RPC for large files
  • [HOPS-22] - StorageCallPrevented Exception must not be swallowed
  • [HOPS-26] - Clean up old MetadataLog code in TransactionalRequestHandler
  • [HOPS-29] - ByteBuffer Mem Leak Problem
  • [HOPS-30] - Expose Parameters For TX Retry
  • [HOPS-31] - Add support to GPUs for hops-metadata-dal
  • [HOPS-32] - Add support to GPUs for hops-metadata-dal-impl-ndb
  • [HOPS-34] - HdfsFileStatus should return the fileId
  • [HOPS-35] - Optimizing logging of MetaEnabled directories
  • [
Read more

2.8.2.2.rc1

16 Nov 10:03
Compare
Choose a tag to compare
2.8.2.2.rc1 Pre-release
Pre-release
    Release notes - hops - Version 2.8.2.2

Bug

  • [HOPS-95] - Differentiate certificate localization directory for RM and NMs
  • [HOPS-96] - Import HDFS-6570 to fix checkAccess issues with Hive
  • [HOPS-99] - Add makeQualified method to HdfsFileStatus and HdfsLocatedFileStatus
  • [HOPS-100] - Fix ssl-server.xml path in HopsSSLSocketFactory
  • [HOPS-102] - Erasure Coding Tests are failing because of excessive memory use
  • [HOPS-104] - Missing check-refresh causes class-cast exception on block report of UC blocks.
  • [HOPS-120] - TestErasureCodingManager regularly run out of memory
  • [HOPS-123] - Guard for no data in phantom blocks
  • [HOPS-127] - pom <repositories> and <distributionManagement> badly configured
  • [HOPS-128] - libhopsyarn pointing to wrong table
  • [HOPS-129] - The update schema script from 2.8.2.1 to 2.8.2.2 should update yarn_price_multiplicator
  • [HOPS-130] - Container log remaining in DB for containers stoped just after creation

New Feature

  • [HOPS-98] - Quota for GPU hours
  • [HOPS-124] - libhopsyarn and libhopsnvml shouldn't be included in the pom files.

Improvement

  • [HOPS-94] - Improve error handling for HopsSSLSocketFactory
  • [HOPS-97] - Use logicaltime per inode to order logs on the same file
  • [HOPS-105] - Parallellize add-stored block after calculating report diff to improve reporting performance
  • [HOPS-119] - Add certificate password as container local resource
  • [HOPS-122] - Recovering ssl certificate info
  • [HOPS-125] - Cleanup hops git repo from hs_err*.log files

2.8.2.1.rc2

16 Nov 10:00
Compare
Choose a tag to compare
2.8.2.1.rc2 Pre-release
Pre-release
    Release notes - hops - Version 2.8.2.1

Bug

  • [HOPS-14] - remove thread in NamenodeSelector
  • [HOPS-16] - Transactions deadlock exception is thrown when hundreds of clients simultaneously read the same file.
  • [HOPS-20] - Lease manager dies on Transient exception.
  • [HOPS-21] - Poor Synchronous implementation of Namenode Selector in HopsFS 2.7.3 causes 80-90% drop in throughput
  • [HOPS-45] - Slow LS operation on directories with large number of immediate descendants
  • [HOPS-47] - TestUsersGroups fail due to wrong order of list elements
  • [HOPS-48] - FATAL UniqueKeyConstraintViolationException During Namenode Creation
  • [HOPS-51] - hdfs native_tests failing
  • [HOPS-56] - setOwner fails with ForeignKeyConstraintViolationException due to outdated cache
  • [HOPS-59] - HopsSSLSocketFactory assumes there is a single proxy superuser
  • [HOPS-67] - Incoming data are not processed in Server
  • [HOPS-69] - Delete directoy does not work non leader namenode when quota is enabled
  • [HOPS-71] - Fix permission issue when writing to Cgroup device subsystem
  • [HOPS-75] - Fix Log Aggregation in Yarn
  • [HOPS-76] - ProcfsBasedProcessTree cannot parse PID process tree info
  • [HOPS-81] - Missing version specifier in hadoop-common dependency causes warnings
  • [HOPS-82] - Hops-GPU should handle cases when a subset of GPUs are not working correctly
  • [HOPS-84] - hadoop-common pom.xml must include hops-gpu-management as dependency
  • [HOPS-85] - Change collate for all the remaining yarn_* tables
  • [HOPS-89] - Owner not correctly set in Cgroup hierarchy
  • [HOPS-90] - Cgroups broken due to bug in container-executor
  • [HOPS-91] - DFSClient update NN list as superuser
  • [HOPS-92] - Symbol lookup error crashes NodeManager
  • [HOPS-93] - Refactor hops-gpu-manage and hops-gpu-management-impl-nvidia
  • [HOPS-103] - Concurrency bug in blockreporting causes iterator infinite loop
  • [HOPS-107] - Small file's read fails when a decommissioned node is selected as a phanton datanode
  • [HOPS-108] - Get HDFS username from environment variable in libhdfs
  • [HOPS-110] - Change column type of small files tables from varchar to varbinary
  • [HOPS-111] - Support set replication api for small files
  • [HOPS-112] - Implement seek api for small files
  • [HOPS-114] - Can't build without -Pndb since [HOPS-103]
  • [HOPS-116] - Spark applications fail when small fiels are read using read(ByteBuffer) api
  • [HOPS-123] - Guard for no data in phantom blocks
  • [HOPS-127] - pom <repositories> and <distributionManagement> badly configured
  • [HOPS-128] - libhopsyarn pointing to wrong table
  • [HOPS-130] - Container log remaining in DB for containers stoped just after creation

Sub-task

  • [HOPS-41] - NPE in Rename for metaEnabled dirs and subdirs
  • [HOPS-42] - Optimize logging of rename in the same meta enabled directory
  • [HOPS-43] - Delete after rename of a metaEnabled directory fail
  • [HOPS-55] - Refactoring block report processing
  • [HOPS-58] - Correcting code quality issues in FSNamesystem
  • [HOPS-60] - Remove remains of Federation from HopsFS
  • [HOPS-62] - Replace Block Report RPC with Bucketized RPC
  • [HOPS-63] - Create a HashBucket entity
  • [HOPS-64] - Replace the block reporting mechanism with the hash-based one

New Feature

  • [HOPS-2] - Improve Small File Performance in HopsFS
  • [HOPS-4] - Uber style resource price
  • [HOPS-7] - TLS support for Hadoop RPC/IPC
  • [HOPS-11] - GPU support for HopsYARN (Umbrella)
  • [HOPS-12] - Create an abstraction layer for GPU devices
  • [HOPS-13] - Create an implementation layer for GPU devices
  • [HOPS-19] - Truncate support in HDFS
  • [HOPS-23] - avoid running app to be able to access other app data
  • [HOPS-124] - libhopsyarn and libhopsnvml shouldn't be included in the pom files.

Improvement

  • [HOPS-17] - Optimizing addBlock and incrementalBlockReport RPC for large files
  • [HOPS-22] - StorageCallPrevented Exception must not be swallowed
  • [HOPS-26] - Clean up old MetadataLog code in TransactionalRequestHandler
  • [HOPS-29] - ByteBuffer Mem Leak Problem
  • [HOPS-30] - Expose Parameters For TX Retry
  • [HOPS-31] - Add support to GPUs for hops-metadata-dal
  • [HOPS-32] - Add support to GPUs for hops-metadata-dal-impl-ndb
  • [HOPS-34] - HdfsFileStatus should return the fileId
  • [HOPS-35] - Optimizing logging of MetaEnabled directories
  • [HOPS-37] - Append To file using HDFS clients (version >= v2.4) fails due to NPE on the HDFS clinet side
  • [HOPS-49] - Making HopsFS and the DAL Java 8 syntax compliant
  • [HOPS-54] - Remove unnecessary debug printouts from dal-impl-ndb
  • ...
Read more

2.8.2.2.rc0

10 Nov 10:06
Compare
Choose a tag to compare
2.8.2.2.rc0 Pre-release
Pre-release
    Release notes - hops - Version 2.8.2.2

Bug

  • [HOPS-95] - Differentiate certificate localization directory for RM and NMs
  • [HOPS-96] - Import HDFS-6570 to fix checkAccess issues with Hive
  • [HOPS-99] - Add makeQualified method to HdfsFileStatus and HdfsLocatedFileStatus
  • [HOPS-100] - Fix ssl-server.xml path in HopsSSLSocketFactory
  • [HOPS-102] - Erasure Coding Tests are failing because of excessive memory use
  • [HOPS-104] - Missing check-refresh causes class-cast exception on block report of UC blocks.
  • [HOPS-120] - TestErasureCodingManager regularly run out of memory
  • [HOPS-123] - Guard for no data in phantom blocks

New Feature

  • [HOPS-98] - Quota for GPU hours
  • [HOPS-124] - libhopsyarn and libhopsnvml shouldn't be included in the pom files.

Improvement

  • [HOPS-94] - Improve error handling for HopsSSLSocketFactory
  • [HOPS-97] - Use logicaltime per inode to order logs on the same file
  • [HOPS-105] - Parallellize add-stored block after calculating report diff to improve reporting performance
  • [HOPS-119] - Add certificate password as container local resource
  • [HOPS-122] - Recovering ssl certificate info
  • [HOPS-125] - Cleanup hops git repo from hs_err*.log files

2.8.2.1.rc1

20 Oct 09:58
Compare
Choose a tag to compare
2.8.2.1.rc1 Pre-release
Pre-release
    Release notes - hops - Version 2.8.2.1

Bug

  • [HOPS-14] - remove thread in NamenodeSelector
  • [HOPS-16] - Transactions deadlock exception is thrown when hundreds of clients simultaneously read the same file.
  • [HOPS-20] - Lease manager dies on Transient exception.
  • [HOPS-21] - Poor Synchronous implementation of Namenode Selector in HopsFS 2.7.3 causes 80-90% drop in throughput
  • [HOPS-45] - Slow LS operation on directories with large number of immediate descendants
  • [HOPS-47] - TestUsersGroups fail due to wrong order of list elements
  • [HOPS-48] - FATAL UniqueKeyConstraintViolationException During Namenode Creation
  • [HOPS-51] - hdfs native_tests failing
  • [HOPS-56] - setOwner fails with ForeignKeyConstraintViolationException due to outdated cache
  • [HOPS-59] - HopsSSLSocketFactory assumes there is a single proxy superuser
  • [HOPS-67] - Incoming data are not processed in Server
  • [HOPS-69] - Delete directoy does not work non leader namenode when quota is enabled
  • [HOPS-71] - Fix permission issue when writing to Cgroup device subsystem
  • [HOPS-75] - Fix Log Aggregation in Yarn
  • [HOPS-76] - ProcfsBasedProcessTree cannot parse PID process tree info
  • [HOPS-81] - Missing version specifier in hadoop-common dependency causes warnings
  • [HOPS-82] - Hops-GPU should handle cases when a subset of GPUs are not working correctly
  • [HOPS-84] - hadoop-common pom.xml must include hops-gpu-management as dependency
  • [HOPS-85] - Change collate for all the remaining yarn_* tables
  • [HOPS-89] - Owner not correctly set in Cgroup hierarchy
  • [HOPS-90] - Cgroups broken due to bug in container-executor
  • [HOPS-91] - DFSClient update NN list as superuser
  • [HOPS-92] - Symbol lookup error crashes NodeManager
  • [HOPS-93] - Refactor hops-gpu-manage and hops-gpu-management-impl-nvidia
  • [HOPS-103] - Concurrency bug in blockreporting causes iterator infinite loop
  • [HOPS-107] - Small file's read fails when a decommissioned node is selected as a phanton datanode
  • [HOPS-108] - Get HDFS username from environment variable in libhdfs
  • [HOPS-111] - Support set replication api for small files
  • [HOPS-112] - Implement seek api for small files
  • [HOPS-114] - Can't build without -Pndb since [HOPS-103]

Sub-task

  • [HOPS-41] - NPE in Rename for metaEnabled dirs and subdirs
  • [HOPS-42] - Optimize logging of rename in the same meta enabled directory
  • [HOPS-43] - Delete after rename of a metaEnabled directory fail
  • [HOPS-55] - Refactoring block report processing
  • [HOPS-58] - Correcting code quality issues in FSNamesystem
  • [HOPS-60] - Remove remains of Federation from HopsFS
  • [HOPS-62] - Replace Block Report RPC with Bucketized RPC
  • [HOPS-63] - Create a HashBucket entity
  • [HOPS-64] - Replace the block reporting mechanism with the hash-based one

New Feature

  • [HOPS-2] - Improve Small File Performance in HopsFS
  • [HOPS-4] - Uber style resource price
  • [HOPS-7] - TLS support for Hadoop RPC/IPC
  • [HOPS-11] - GPU support for HopsYARN (Umbrella)
  • [HOPS-12] - Create an abstraction layer for GPU devices
  • [HOPS-13] - Create an implementation layer for GPU devices
  • [HOPS-19] - Truncate support in HDFS
  • [HOPS-23] - avoid running app to be able to access other app data

Improvement

  • [HOPS-17] - Optimizing addBlock and incrementalBlockReport RPC for large files
  • [HOPS-22] - StorageCallPrevented Exception must not be swallowed
  • [HOPS-26] - Clean up old MetadataLog code in TransactionalRequestHandler
  • [HOPS-29] - ByteBuffer Mem Leak Problem
  • [HOPS-30] - Expose Parameters For TX Retry
  • [HOPS-31] - Add support to GPUs for hops-metadata-dal
  • [HOPS-32] - Add support to GPUs for hops-metadata-dal-impl-ndb
  • [HOPS-34] - HdfsFileStatus should return the fileId
  • [HOPS-35] - Optimizing logging of MetaEnabled directories
  • [HOPS-37] - Append To file using HDFS clients (version >= v2.4) fails due to NPE on the HDFS clinet side
  • [HOPS-49] - Making HopsFS and the DAL Java 8 syntax compliant
  • [HOPS-54] - Remove unnecessary debug printouts from dal-impl-ndb
  • [HOPS-57] - Hash based block-reporting for improved block reporting speed
  • [HOPS-61] - Block Management Refactoring
  • [HOPS-65] - update to follow hadoop 2.8.2 updates
  • [HOPS-66] - Add a script to upgrade Clusterj with hopsfs fixes and Clusterj Native in Kompics repo
  • [HOPS-68] - Pass the keystore and truststore password along with the material
  • [HOPS-79] - Apply patch YARN-7083 from upstream fixing Log Aggregation
  • [HOPS-83] - Cgroups hierarchy should be created automatically

2.8.2.1.rc0

19 Sep 15:36
Compare
Choose a tag to compare
2.8.2.1.rc0 Pre-release
Pre-release
   Release notes - hops - Version 2.8.2.1

Bug

  • [HOPS-14] - remove thread in NamenodeSelector
  • [HOPS-16] - Transactions deadlock exception is thrown when hundreds of clients simultaneously read the same file.
  • [HOPS-20] - Lease manager dies on Transient exception.
  • [HOPS-21] - Poor Synchronous implementation of Namenode Selector in HopsFS 2.7.3 causes 80-90% drop in throughput
  • [HOPS-45] - Slow LS operation on directories with large number of immediate descendants
  • [HOPS-47] - TestUsersGroups fail due to wrong order of list elements
  • [HOPS-48] - FATAL UniqueKeyConstraintViolationException During Namenode Creation
  • [HOPS-51] - hdfs native_tests failing
  • [HOPS-56] - setOwner fails with ForeignKeyConstraintViolationException due to outdated cache
  • [HOPS-59] - HopsSSLSocketFactory assumes there is a single proxy superuser
  • [HOPS-67] - Incoming data are not processed in Server
  • [HOPS-69] - Delete directoy does not work non leader namenode when quota is enabled
  • [HOPS-71] - Fix permission issue when writing to Cgroup device subsystem
  • [HOPS-75] - Fix Log Aggregation in Yarn
  • [HOPS-76] - ProcfsBasedProcessTree cannot parse PID process tree info
  • [HOPS-78] - Fix TestFsck test
  • [HOPS-81] - Missing version specifier in hadoop-common dependency causes warnings
  • [HOPS-82] - Hops-GPU should handle cases when a subset of GPUs are not working correctly
  • [HOPS-84] - hadoop-common pom.xml must include hops-gpu-management as dependency
  • [HOPS-85] - Change collate for all the remaining yarn_* tables
  • [HOPS-89] - Owner not correctly set in Cgroup hierarchy

Sub-task

  • [HOPS-41] - NPE in Rename for metaEnabled dirs and subdirs
  • [HOPS-42] - Optimize logging of rename in the same meta enabled directory
  • [HOPS-43] - Delete after rename of a metaEnabled directory fail
  • [HOPS-55] - Refactoring block report processing
  • [HOPS-58] - Correcting code quality issues in FSNamesystem
  • [HOPS-60] - Remove remains of Federation from HopsFS
  • [HOPS-62] - Replace Block Report RPC with Bucketized RPC
  • [HOPS-63] - Create a HashBucket entity
  • [HOPS-64] - Replace the block reporting mechanism with the hash-based one

New Feature

  • [HOPS-2] - Improve Small File Performance in HopsFS
  • [HOPS-4] - Uber style resource price
  • [HOPS-7] - TLS support for Hadoop RPC/IPC
  • [HOPS-11] - GPU support for HopsYARN (Umbrella)
  • [HOPS-12] - Create an abstraction layer for GPU devices
  • [HOPS-13] - Create an implementation layer for GPU devices
  • [HOPS-19] - Truncate support in HDFS
  • [HOPS-23] - avoid running app to be able to access other app data

Improvement

  • [HOPS-17] - Optimizing addBlock and incrementalBlockReport RPC for large files
  • [HOPS-22] - StorageCallPrevented Exception must not be swallowed
  • [HOPS-26] - Clean up old MetadataLog code in TransactionalRequestHandler
  • [HOPS-29] - ByteBuffer Mem Leak Problem
  • [HOPS-30] - Expose Parameters For TX Retry
  • [HOPS-31] - Add support to GPUs for hops-metadata-dal
  • [HOPS-32] - Add support to GPUs for hops-metadata-dal-impl-ndb
  • [HOPS-34] - HdfsFileStatus should return the fileId
  • [HOPS-35] - Optimizing logging of MetaEnabled directories
  • [HOPS-37] - Append To file using HDFS clients (version >= v2.4) fails due to NPE on the HDFS clinet side
  • [HOPS-49] - Making HopsFS and the DAL Java 8 syntax compliant
  • [HOPS-54] - Remove unnecessary debug printouts from dal-impl-ndb
  • [HOPS-57] - Hash based block-reporting for improved block reporting speed
  • [HOPS-61] - Block Management Refactoring
  • [HOPS-65] - update to follow hadoop 2.8.2 updates
  • [HOPS-66] - Add a script to upgrade Clusterj with hopsfs fixes and Clusterj Native in Kompics repo
  • [HOPS-68] - Pass the keystore and truststore password along with the material
  • [HOPS-79] - Apply patch YARN-7083 from upstream fixing Log Aggregation
  • [HOPS-83] - Cgroups hierarchy should be created automatically