Skip to content

update with pushgateway updated libs#2

Open
casmssm wants to merge 1 commit into
dinumathai:masterfrom
casmssm:fix/fatal_error_concurrent_map_iteration_and_map_write
Open

update with pushgateway updated libs#2
casmssm wants to merge 1 commit into
dinumathai:masterfrom
casmssm:fix/fatal_error_concurrent_map_iteration_and_map_write

Conversation

@casmssm

@casmssm casmssm commented Jul 11, 2022

Copy link
Copy Markdown

Hi, recently We got this error "fatal error: concurrent map iteration and map write".

I work at Globo.com and we tested the new compiled version of your modified pushgateway (TTL) with new libs from pushgateway. It's working until now.

Then I sent this pull request.

Thanks for your contribution.

@menghe999

menghe999 commented Mar 14, 2023

Copy link
Copy Markdown

Hi @casmssm have you solved the problems ? Now,i using your branch fix/fatal_error_concurrent_map_iteration_and_map_write,and met the same problem. But i don't konw how to fix.

the flow is part of log stack

ts=2023-02-28T07:41:44.787Z caller=main.go:100 level=info msg="starting pushgateway" version="(version=1.5.1, branch=HEAD, revision=7afc96cfc5e2b278f3b20e56968ff30eea22b70e)"
ts=2023-02-28T07:41:44.788Z caller=main.go:101 level=info build_context="(go=go1.19.3, user=root@fc81889ee1a6, date=20221129-16:30:38)"
ts=2023-02-28T07:41:44.791Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9091
ts=2023-02-28T07:41:44.791Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9091
level=info ts=2023-03-07T09:54:11.778Z caller=main.go:86 msg="starting pushgateway" version="(version=, branch=, revision=)"
level=info ts=2023-03-07T09:54:11.778Z caller=main.go:87 build_context="(go=go1.20.1, user=, date=)"
level=info ts=2023-03-07T09:54:11.783Z caller=main.go:140 listen_address=:9091
level=info ts=2023-03-07T09:54:11.783Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
fatal error: concurrent map iteration and map write

goroutine 127 [running]:
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).GetMetricFamiliesMap(0xc0001aa840)
	/home/mengh/pgw/pushgateway-ttl-1.4.0/storage/diskmetricstore.go:199 +0x24c
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).checkWriteRequest(0xc0001aa840, {0xc000dc9bf0, {0xc0f9f4c406a2e169, 0x130b8ded8bf7, 0x1023080}, 0xc000dc9c50, 0x1, 0xc001df4cc0})
	/home/mengh/pgw/pushgateway-ttl-1.4.0/storage/diskmetricstore.go:377 +0x259
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).loop(0xc0001aa840, 0x45d964b800)
	/home/mengh/pgw/pushgateway-ttl-1.4.0/storage/diskmetricstore.go:235 +0x2dd
created by github.com/prometheus/pushgateway/storage.NewDiskMetricStore
	/home/mengh/pgw/pushgateway-ttl-1.4.0/storage/diskmetricstore.go:104 +0x4ac

goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x7f11e125a018, 0x72)
	/home/mengh/pgw/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000160180?, 0x8?, 0x0)
	/home/mengh/pgw/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/home/mengh/pgw/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000160180)
	/home/mengh/pgw/go/src/internal/poll/fd_unix.go:614 +0x2bd
net.(*netFD).accept(0xc000160180)
	/home/mengh/pgw/go/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc000013f20)
	/home/mengh/pgw/go/src/net/tcpsock_posix.go:148 +0x25
net.(*TCPListener).Accept(0xc000013f20)
	/home/mengh/pgw/go/src/net/tcpsock.go:297 +0x3d
net/http.(*Server).Serve(0xc0002631d0, {0xc8baf0, 0xc000013f20})
	/home/mengh/pgw/go/src/net/http/server.go:3059 +0x385
github.com/prometheus/exporter-toolkit/web.Serve({0xc8baf0, 0xc000013f20}, 0xc0002631d0, {0x0?, 0xc00076a300?}, {0xc87fe0?, 0xc000276a50?})
	/root/go/pkg/mod/github.com/prometheus/exporter-toolkit@v0.5.1/web/tls_config.go:192 +0x72d
main.main()
	/home/mengh/pgw/pushgateway-ttl-1.4.0/main.go:199 +0x343d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants