opensearch won't start in local docker env at first time #1761

sophiaxu0424 · 2024-10-30T21:30:00Z

Bug Report

Which version of the demo you are using? (please provide either a specific

main branch: https://github.com/open-telemetry/opentelemetry-demo/commits/main

and hash: bafadb0

Symptom

A clear and concise description of what the bug is.

What is the expected behavior?
opensearch container is up and running

What do you expect to see?
opensearch container is up and running successfully

What is the actual behavior?

[2024-10-30T21:25:35,091][ERROR][o.o.s.l.BuiltinLogTypeLoader] [demo-node] Failed loading builtin log types from disk!
java.nio.file.FileSystemNotFoundException: null
	at [email protected]/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:156) ~[?:?]
	at [email protected]/jdk.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:142) ~[?:?]
	at java.base/java.nio.file.Path.of(Path.java:209) ~[?:?]
	at java.base/java.nio.file.Paths.get(Paths.java:98) ~[?:?]
	at org.opensearch.securityanalytics.logtype.BuiltinLogTypeLoader.loadBuiltinLogTypes(BuiltinLogTypeLoader.java:73) ~[opensearch-security-analytics-2.17.1.0.jar:2.17.1.0]
	at org.opensearch.securityanalytics.logtype.BuiltinLogTypeLoader.ensureLogTypesLoaded(BuiltinLogTypeLoader.java:62) [opensearch-security-analytics-2.17.1.0.jar:2.17.1.0]
	at org.opensearch.securityanalytics.logtype.BuiltinLogTypeLoader.doStart(BuiltinLogTypeLoader.java:146) [opensearch-security-analytics-2.17.1.0.jar:2.17.1.0]
	at org.opensearch.common.lifecycle.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:77) [opensearch-common-2.17.1.jar:2.17.1]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) [?:?]

Please describe the actual behavior experienced.

Reproduce

Could you provide the minimum required steps to resolve the issue you're seeing?

make start

or

docker-compose up --force-recreate --remove-orphans --detach

We will close this issue if:

The steps you provided are complex.
If we can not reproduce the behavior you're reporting.

Additional Context

Please feel free to add any other context about the problem here.

Re-start the the containers again and again, would get a healthy status in opensearch.

The text was updated successfully, but these errors were encountered:

julianocosta89 · 2024-10-31T07:34:15Z

Hello @sophiaxu0424, which OS are you running the demo?
Does the error also happen when you run make start?
I saw that you also suggested docker-compose, but that won't work. We use docker compose, the version with the - is the old compose version and not supported.

sophiaxu0424 · 2024-10-31T15:07:30Z

Hi @julianocosta89 i am using a MAC OS with M1, I saw that the docker-compose has depreciated, but it might because of my local setting in /usr/local/bin, it is still using docker-compose command with the latest version.

julianocosta89 · 2024-11-04T07:24:44Z

@sophiaxu0424 if you have the latest version of Docker installed, it already supports docker compose.
Could you try the following and let us know if it helps?

docker compose down
docker compose up --force-recreate --remove-orphans --detach

sophiaxu0424 · 2024-11-08T16:59:19Z

Still having this error

sophiaxu@Sophias-MBP-2 opentelemetry-demo % docker compose down
docker compose up --force-recreate --remove-orphans --detach
[+] Running 26/26
 ✔ Container accounting-service       Removed                                                                                                                      0.2s
 ✔ Container prometheus               Removed                                                                                                                      2.6s
 ✔ Container frauddetection-service   Removed                                                                                                                      0.2s
 ✔ Container frontend-proxy           Removed                                                                                                                      0.2s
 ✔ Container grafana                  Removed                                                                                                                      2.9s
 ✔ Container flagd-ui                 Removed                                                                                                                      0.1s
 ✔ Container load-generator           Removed                                                                                                                      0.1s
 ✔ Container frontend                 Removed                                                                                                                      0.1s
 ✔ Container imageprovider            Removed                                                                                                                      0.1s
 ✔ Container checkout-service         Removed                                                                                                                      0.1s
 ✔ Container ad-service               Removed                                                                                                                      0.2s
 ✔ Container recommendation-service   Removed                                                                                                                      0.2s
 ✔ Container quote-service            Removed                                                                                                                      0.2s
 ✔ Container payment-service          Removed                                                                                                                      0.2s
 ✔ Container kafka                    Removed                                                                                                                      3.6s
 ✔ Container currency-service         Removed                                                                                                                      0.2s
 ✔ Container cart-service             Removed                                                                                                                      0.2s
 ✔ Container email-service            Removed                                                                                                                      0.2s
 ✔ Container shipping-service         Removed                                                                                                                      0.2s
 ✔ Container product-catalog-service  Removed                                                                                                                      0.3s
 ✔ Container valkey-cart              Removed                                                                                                                      1.9s
 ✔ Container otel-col                 Removed                                                                                                                      0.2s
 ✔ Container flagd                    Removed                                                                                                                      1.8s
 ✔ Container jaeger                   Removed                                                                                                                      2.0s
 ✔ Container opensearch               Removed                                                                                                                      1.9s
 ✔ Network opentelemetry-demo         Removed                                                                                                                      0.2s
[+] Running 26/26
 ✔ Network opentelemetry-demo         Created                                                                                                                      0.1s
 ✘ Container opensearch               Error                                                                                                                      278.8s
 ✔ Container prometheus               Started                                                                                                                      2.0s
 ✔ Container grafana                  Started                                                                                                                      2.0s
 ✔ Container jaeger                   Started                                                                                                                      2.2s
 ✔ Container kafka                    Started                                                                                                                      2.2s
 ✔ Container valkey-cart              Started                                                                                                                      2.0s
 ✔ Container flagd                    Started                                                                                                                      2.2s
 ✔ Container otel-col                 Created                                                                                                                      0.1s
 ✔ Container accounting-service       Created                                                                                                                      0.2s
 ✔ Container imageprovider            Created                                                                                                                      0.2s
 ✔ Container payment-service          Created                                                                                                                      0.2s
 ✔ Container quote-service            Created                                                                                                                      0.2s
 ✔ Container cart-service             Created                                                                                                                      0.2s
 ✔ Container frauddetection-service   Created                                                                                                                      0.3s
 ✔ Container flagd-ui                 Created                                                                                                                      0.2s
 ✔ Container currency-service         Created                                                                                                                      0.2s
 ✔ Container ad-service               Created                                                                                                                      0.2s
 ✔ Container product-catalog-service  Created                                                                                                                      0.2s
 ✔ Container shipping-service         Created                                                                                                                      0.2s
 ✔ Container email-service            Created                                                                                                                      0.2s
 ✔ Container recommendation-service   Created                                                                                                                      0.1s
 ✔ Container checkout-service         Created                                                                                                                      0.1s
 ✔ Container frontend                 Created                                                                                                                      0.1s
 ✔ Container load-generator           Created                                                                                                                      0.1s
 ✔ Container frontend-proxy           Created                                                                                                                      0.1s
dependency failed to start: container opensearch is unhealthy

sophiaxu0424 · 2024-11-08T17:15:07Z

and docker logs about this opensearch container

sophiaxu@Sophias-MBP-2 opentelemetry-demo % docker logs 549022bed7c3
Disabling OpenSearch Security Plugin
Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.17.1.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
Nov 08, 2024 4:02:55 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.17.1.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2024-11-08T16:02:58,363][INFO ][o.o.n.Node               ] [demo-node] version[2.17.1], pid[1], build[tar/1893d20797e30110e5877170e44d42275ce5951e/2024-09-26T21:59:52.691008096Z], OS[Linux/6.5.0-44-generic/aarch64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/21.0.4/21.0.4+7-LTS]
[2024-11-08T16:02:58,382][INFO ][o.o.n.Node               ] [demo-node] JVM home [/usr/share/opensearch/jdk], using bundled JDK/JRE [true]
[2024-11-08T16:02:58,383][INFO ][o.o.n.Node               ] [demo-node] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-12179628951096343081, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, --add-modules=jdk.incubator.vector, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms300m, -Xmx300m, -XX:MaxDirectMemorySize=157286400, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2024-11-08T16:03:00,047][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [demo-node] Java vector incubator API enabled; uses preferredBitSize=128; FMA enabled
[2024-11-08T16:03:06,826][INFO ][o.o.s.s.t.SSLConfig      ] [demo-node] SSL dual mode is disabled
[2024-11-08T16:03:06,837][WARN ][o.o.s.OpenSearchSecurityPlugin] [demo-node] OpenSearch Security plugin installed but disabled. This can expose your configuration (including passwords) to the public.
[2024-11-08T16:03:09,527][INFO ][o.o.p.c.c.PluginSettings ] [demo-node] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
[2024-11-08T16:03:12,547][INFO ][o.o.i.r.ReindexPlugin    ] [demo-node] ReindexPlugin reloadSPI called
[2024-11-08T16:03:12,550][INFO ][o.o.i.r.ReindexPlugin    ] [demo-node] Unable to find any implementation for RemoteReindexExtension
[2024-11-08T16:03:12,843][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions
[2024-11-08T16:03:12,845][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: opensearch_time_series_analytics, index: .opendistro-anomaly-detector-jobs
[2024-11-08T16:03:12,861][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config
[2024-11-08T16:03:12,877][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: scheduler_geospatial_ip2geo_datasource, index: .scheduler-geospatial-ip2geo-datasource
[2024-11-08T16:03:12,879][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: opensearch_sap_job, index: .opensearch-sap--job
[2024-11-08T16:03:12,890][INFO ][o.o.j.JobSchedulerPlugin ] [demo-node] Loaded scheduler extension: async-query-scheduler, index: .async-query-scheduler
[2024-11-08T16:03:13,005][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [aggs-matrix-stats]
[2024-11-08T16:03:13,005][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [analysis-common]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [cache-common]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [geo]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [ingest-common]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [ingest-geoip]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [ingest-user-agent]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [lang-expression]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [lang-mustache]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [lang-painless]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [mapper-extras]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [opensearch-dashboards]
[2024-11-08T16:03:13,006][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [parent-join]
[2024-11-08T16:03:13,007][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [percolator]
[2024-11-08T16:03:13,007][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [rank-eval]
[2024-11-08T16:03:13,008][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [reindex]
[2024-11-08T16:03:13,014][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [repository-url]
[2024-11-08T16:03:13,019][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [search-pipeline-common]
[2024-11-08T16:03:13,019][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [systemd]
[2024-11-08T16:03:13,019][INFO ][o.o.p.PluginsService     ] [demo-node] loaded module [transport-netty4]
[2024-11-08T16:03:13,020][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-alerting]
[2024-11-08T16:03:13,026][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-anomaly-detection]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-asynchronous-search]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-cross-cluster-replication]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-custom-codecs]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-flow-framework]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-geospatial]
[2024-11-08T16:03:13,028][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-index-management]
[2024-11-08T16:03:13,029][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-job-scheduler]
[2024-11-08T16:03:13,042][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-knn]
[2024-11-08T16:03:13,042][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-ml]
[2024-11-08T16:03:13,042][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-neural-search]
[2024-11-08T16:03:13,043][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-notifications]
[2024-11-08T16:03:13,043][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-notifications-core]
[2024-11-08T16:03:13,043][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-observability]
[2024-11-08T16:03:13,043][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-performance-analyzer]
[2024-11-08T16:03:13,064][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-reports-scheduler]
[2024-11-08T16:03:13,064][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-security]
[2024-11-08T16:03:13,064][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-security-analytics]
[2024-11-08T16:03:13,064][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-skills]
[2024-11-08T16:03:13,064][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-sql]
[2024-11-08T16:03:13,066][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [opensearch-system-templates]
[2024-11-08T16:03:13,066][INFO ][o.o.p.PluginsService     ] [demo-node] loaded plugin [query-insights]
[2024-11-08T16:03:13,496][WARN ][stderr                   ] [demo-node] WARNING: A restricted method in java.lang.foreign.Linker has been called
[2024-11-08T16:03:13,496][WARN ][stderr                   ] [demo-node] WARNING: java.lang.foreign.Linker::downcallHandle has been called by the unnamed module
[2024-11-08T16:03:13,496][WARN ][stderr                   ] [demo-node] WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for this module
[2024-11-08T16:03:13,662][INFO ][o.a.l.s.MemorySegmentIndexInputProvider] [demo-node] Using MemorySegmentIndexInput and native madvise support with Java 21 or later; to disable start with -Dorg.apache.lucene.store.MMapDirectory.enableMemorySegments=false
[2024-11-08T16:03:13,679][INFO ][o.o.e.NodeEnvironment    ] [demo-node] using [1] data paths, mounts [[/ (overlay)]], net usable_space [38.4gb], net total_space [57gb], types [overlay]
[2024-11-08T16:03:13,679][INFO ][o.o.e.NodeEnvironment    ] [demo-node] heap size [300mb], compressed ordinary object pointers [true]
[2024-11-08T16:03:13,842][INFO ][o.o.n.Node               ] [demo-node] node name [demo-node], node ID [UsVAuE0KSX-0jUCgTlPjYw], cluster name [demo-cluster], roles [ingest, remote_cluster_client, data, cluster_manager]
[2024-11-08T16:03:14,107][INFO ][o.o.e.ExtensionsManager  ] [demo-node] ExtensionsManager initialized
[2024-11-08T16:03:27,754][DEPRECATION][o.o.d.c.s.Settings       ] [demo-node] [index.store.hybrid.mmap.extensions] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version.
[2024-11-08T16:03:46,865][INFO ][o.o.n.p.NeuralSearch     ] [demo-node] Registering hybrid query phase searcher with feature flag [plugins.neural_search.hybrid_search_disabled]
[2024-11-08T16:04:29,675][INFO ][o.o.t.b.CircuitBreakerService] [demo-node] Registered memory breaker.
[2024-11-08T16:09:36,285][INFO ][o.o.m.b.MLCircuitBreakerService] [demo-node] Registered ML memory breaker.
[2024-11-08T16:09:36,435][INFO ][o.o.m.b.MLCircuitBreakerService] [demo-node] Registered ML disk breaker.
[2024-11-08T16:09:36,435][INFO ][o.o.m.b.MLCircuitBreakerService] [demo-node] Registered ML native memory breaker.
[2024-11-08T16:11:27,707][INFO ][o.r.Reflections          ] [demo-node] Reflections took 7533 ms to scan 1 urls, producing 26 keys and 67 values
[2024-11-08T16:11:32,640][INFO ][o.r.Reflections          ] [demo-node] Reflections took 264 ms to scan 1 urls, producing 3 keys and 5 values
[2024-11-08T16:12:12,530][WARN ][o.o.s.p.SQLPlugin        ] [demo-node] Master key is a required config for using create and update datasource APIs. Please set plugins.query.datasources.encryption.masterkey config in opensearch.yml in all the cluster nodes. More details can be found here: https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/datasources.rst#master-key-config-for-encrypting-credential-information
Disabling OpenSearch Security Plugin
Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.17.1.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
Nov 08, 2024 4:55:29 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
[2024-11-08T16:56:36,633][INFO ][o.o.n.Node               ] [demo-node] version[2.17.1], pid[1], build[tar/1893d20797e30110e5877170e44d42275ce5951e/2024-09-26T21:59:52.691008096Z], OS[Linux/6.5.0-44-generic/aarch64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/21.0.4/21.0.4+7-LTS]
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.17.1.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2024-11-08T16:56:36,811][INFO ][o.o.n.Node               ] [demo-node] JVM home [/usr/share/opensearch/jdk], using bundled JDK/JRE [true]
[2024-11-08T16:56:36,811][INFO ][o.o.n.Node               ] [demo-node] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-17850748487669341561, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, --add-modules=jdk.incubator.vector, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms300m, -Xmx300m, -XX:MaxDirectMemorySize=157286400, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2024-11-08T16:57:10,789][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [demo-node] Java vector incubator API enabled; uses preferredBitSize=128; FMA enabled
Disabling OpenSearch Security Plugin
Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
WARNING: Using incubator modules: jdk.incubator.vector
............
............

julianocosta89 · 2024-11-09T10:06:40Z

@sophiaxu0424 which OS are you running this?
I can't reproduce the error on my end.

Also, we have just release v 1.12.0, could you try pulling the latest changes and run the commands again?

carlyrichmond · 2024-11-18T15:56:40Z

@sophiaxu0424 I had a similar issue today and resolved it by cleaning up dangling volumes using the steps in the below threads:

https://discuss.elastic.co/t/no-space-left-on-device/248400

https://stackoverflow.com/questions/30604846/docker-error-no-space-left-on-device#:~:text=Check%20that%20you%20have%20free,var%2Flib%2Fdocker).&text=If%20you%20edit%20%2Fetc%2Fdefault,the%20change%20to%20take%20effect.

Let me know if that addresses your issue, too.

sophiaxu0424 · 2024-11-20T15:17:07Z

tried command docker system prune and it helped reclaimed back some space for me. but restarting the demo images/containers still having this error.

sophiaxu0424 · 2024-11-20T16:53:37Z

Not sure if it was a conflict about my local docker install, I have two local installation and one with brew, another one with dmg install, we can't use docker desktop, and I'm using colima instead.

After i removed the rm -rf ~/.docker/ and re installed/upgraded again using brew and re-started colima, it is working for now.

sophiaxu0424 · 2024-11-20T16:55:09Z

Thanks for the all the investigation till now, a lot of cmd like docker useage cmds are pretty useful for me. Close this for now

sophiaxu0424 added the bug Something isn't working label Oct 30, 2024

sophiaxu0424 closed this as completed Nov 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

opensearch won't start in local docker env at first time #1761

opensearch won't start in local docker env at first time #1761

sophiaxu0424 commented Oct 30, 2024

julianocosta89 commented Oct 31, 2024

sophiaxu0424 commented Oct 31, 2024

julianocosta89 commented Nov 4, 2024

sophiaxu0424 commented Nov 8, 2024

sophiaxu0424 commented Nov 8, 2024

julianocosta89 commented Nov 9, 2024

carlyrichmond commented Nov 18, 2024

sophiaxu0424 commented Nov 20, 2024

sophiaxu0424 commented Nov 20, 2024

sophiaxu0424 commented Nov 20, 2024

opensearch won't start in local docker env at first time #1761

opensearch won't start in local docker env at first time #1761

Comments

sophiaxu0424 commented Oct 30, 2024

Bug Report

Symptom

Reproduce

Additional Context

julianocosta89 commented Oct 31, 2024

sophiaxu0424 commented Oct 31, 2024

julianocosta89 commented Nov 4, 2024

sophiaxu0424 commented Nov 8, 2024

sophiaxu0424 commented Nov 8, 2024

julianocosta89 commented Nov 9, 2024

carlyrichmond commented Nov 18, 2024

sophiaxu0424 commented Nov 20, 2024

sophiaxu0424 commented Nov 20, 2024

sophiaxu0424 commented Nov 20, 2024