2024-09-03 13:48:36,339 INFO    StreamThr :59204 [internal.py:wandb_internal():86] W&B internal server running at pid: 59204, started at: 2024-09-03 13:48:36.337597
2024-09-03 13:48:36,340 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status
2024-09-03 13:48:36,342 INFO    WriterThread:59204 [datastore.py:open_for_write():87] open: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/run-wise-water-249.wandb
2024-09-03 13:48:36,344 DEBUG   SenderThread:59204 [sender.py:send():379] send: header
2024-09-03 13:48:36,358 DEBUG   SenderThread:59204 [sender.py:send():379] send: run
2024-09-03 13:48:36,375 INFO    SenderThread:59204 [sender.py:_setup_resume():749] checking resume status for tabpfn_interpretability/xlstm-experiments/wise-water-249
2024-09-03 13:48:36,646 INFO    SenderThread:59204 [sender.py:_setup_resume():827] configured resuming with: ResumeState(resumed=True,step=0,history=0,events=16,output=124,runtime=483.982519,wandb_runtime=None,summary={},config={'_wandb': {'desc': None, 'value': {'t': {'1': [1, 11, 49, 55], '3': [14, 23], '4': '3.11.8', '5': '0.16.6', '6': '4.42.4', '8': [5], '13': 'darwin-arm64'}, 'framework': 'huggingface', 'start_time': 1725364093.0, 'cli_version': '0.16.6', 'is_jupyter_run': False, 'python_version': '3.11.8', 'is_kaggle_kernel': False, 'huggingface_version': '4.42.4'}}},tags=[])
2024-09-03 13:48:37,691 INFO    SenderThread:59204 [dir_watcher.py:__init__():211] watching files in: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files
2024-09-03 13:48:37,692 INFO    SenderThread:59204 [sender.py:_start_run_threads():1124] run started: wise-water-249 with start time 1725363632.356426
2024-09-03 13:48:37,703 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: check_version
2024-09-03 13:48:37,703 DEBUG   SenderThread:59204 [sender.py:send_request():406] send_request: check_version
2024-09-03 13:48:37,792 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: run_start
2024-09-03 13:48:37,799 DEBUG   HandlerThread:59204 [system_info.py:__init__():26] System info init
2024-09-03 13:48:37,800 DEBUG   HandlerThread:59204 [system_info.py:__init__():41] System info init done
2024-09-03 13:48:37,800 INFO    HandlerThread:59204 [system_monitor.py:start():194] Starting system monitor
2024-09-03 13:48:37,800 INFO    SystemMonitor:59204 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-09-03 13:48:37,801 INFO    SystemMonitor:59204 [interfaces.py:start():190] Started cpu monitoring
2024-09-03 13:48:37,803 INFO    SystemMonitor:59204 [interfaces.py:start():190] Started disk monitoring
2024-09-03 13:48:37,804 INFO    SystemMonitor:59204 [interfaces.py:start():190] Started gpuapple monitoring
2024-09-03 13:48:37,804 INFO    SystemMonitor:59204 [interfaces.py:start():190] Started memory monitoring
2024-09-03 13:48:37,805 INFO    SystemMonitor:59204 [interfaces.py:start():190] Started network monitoring
2024-09-03 13:48:38,169 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: python_packages
2024-09-03 13:48:38,170 DEBUG   SenderThread:59204 [sender.py:send_request():406] send_request: python_packages
2024-09-03 13:48:38,170 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: stop_status
2024-09-03 13:48:38,171 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: internal_messages
2024-09-03 13:48:38,171 DEBUG   SenderThread:59204 [sender.py:send_request():406] send_request: stop_status
2024-09-03 13:48:38,452 DEBUG   SenderThread:59204 [sender.py:send():379] send: telemetry
2024-09-03 13:48:38,700 INFO    Thread-16 :59204 [dir_watcher.py:_on_file_created():271] file/dir created: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files/output.log
2024-09-03 13:48:38,701 INFO    Thread-16 :59204 [dir_watcher.py:_on_file_created():271] file/dir created: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files/requirements.txt
2024-09-03 13:48:40,705 INFO    Thread-16 :59204 [dir_watcher.py:_on_file_modified():288] file/dir modified: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files/output.log
2024-09-03 13:48:41,462 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:48:45,731 INFO    Thread-16 :59204 [dir_watcher.py:_on_file_modified():288] file/dir modified: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files/output.log
2024-09-03 13:48:47,302 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:48:52,320 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:48:53,175 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: stop_status
2024-09-03 13:48:53,175 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: internal_messages
2024-09-03 13:48:53,176 DEBUG   SenderThread:59204 [sender.py:send_request():406] send_request: stop_status
2024-09-03 13:48:57,365 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:49:02,382 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:49:07,409 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: status_report
2024-09-03 13:49:07,809 INFO    Thread-16 :59204 [dir_watcher.py:_on_file_modified():288] file/dir modified: /Users/siemsj/projects/xlstm/experiments/wandb/run-20240903_134836-wise-water-249/files/config.yaml
2024-09-03 13:49:08,176 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: stop_status
2024-09-03 13:49:08,177 DEBUG   HandlerThread:59204 [handler.py:handle_request():146] handle_request: internal_messages
2024-09-03 13:49:08,177 DEBUG   SenderThread:59204 [sender.py:send_request():406] send_request: stop_status
