FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881425         	|  1587.1         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  77.027         	|  20             	|  1540.5         	|  97.065         	|
|  run_training_batch                                                                                                                                                                                         	|  0.067544       	|  20000          	|  1350.9         	|  85.115         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.067096       	|  20000          	|  1341.9         	|  84.55          	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.021915       	|  20000          	|  438.29         	|  27.615         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0061343      	|  20000          	|  122.69         	|  7.73           	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0050502      	|  20000          	|  101.0          	|  6.3639         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.63714        	|  60             	|  38.228         	|  2.4086         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  35.666         	|  1              	|  35.666         	|  2.2472         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00053395     	|  20000          	|  10.679         	|  0.67285        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.19939        	|  20             	|  3.9878         	|  0.25126        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00018132     	|  20060          	|  3.6374         	|  0.22918        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00011702     	|  20060          	|  2.3474         	|  0.1479         	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.5373e-05     	|  20000          	|  0.70745        	|  0.044574       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.0731e-05     	|  20000          	|  0.41463        	|  0.026124       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.9914e-05     	|  20000          	|  0.39827        	|  0.025094       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.004337       	|  60             	|  0.26022        	|  0.016396       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.059735       	|  2              	|  0.11947        	|  0.0075274      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.9203e-06     	|  20000          	|  0.098407       	|  0.0062003      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.4286e-06     	|  20000          	|  0.088572       	|  0.0055806      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  4.1119e-06     	|  20000          	|  0.082238       	|  0.0051815      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  3.6327e-06     	|  20000          	|  0.072654       	|  0.0045777      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.5732e-06     	|  20000          	|  0.071463       	|  0.0045027      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.2469e-06     	|  20000          	|  0.064938       	|  0.0040915      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.2464e-06     	|  20000          	|  0.064929       	|  0.0040909      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.1927e-06     	|  20000          	|  0.063853       	|  0.0040232      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.0985e-06     	|  20000          	|  0.061971       	|  0.0039046      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  3.0185e-06     	|  20000          	|  0.06037        	|  0.0038037      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.6121e-06     	|  20000          	|  0.052243       	|  0.0032917      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.6055e-06     	|  20000          	|  0.052109       	|  0.0032832      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  2.5253e-06     	|  20060          	|  0.050657       	|  0.0031918      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.4594e-06     	|  20000          	|  0.049188       	|  0.0030992      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.3032e-06     	|  20000          	|  0.046064       	|  0.0029023      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.2996e-06     	|  20000          	|  0.045993       	|  0.0028978      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2833e-06     	|  20000          	|  0.045666       	|  0.0028773      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1975e-06     	|  20000          	|  0.043949       	|  0.0027691      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.1677e-06     	|  20000          	|  0.043355       	|  0.0027316      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1237e-06     	|  20000          	|  0.042474       	|  0.0026761      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.1026e-06     	|  20000          	|  0.042053       	|  0.0026496      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0553e-06     	|  20000          	|  0.041105       	|  0.0025899      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.034e-06      	|  20000          	|  0.04068        	|  0.0025631      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.5059e-06     	|  20060          	|  0.030209       	|  0.0019034      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.4012e-06     	|  20000          	|  0.028025       	|  0.0017658      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.3337e-06     	|  20000          	|  0.026673       	|  0.0016806      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.3103e-06     	|  20000          	|  0.026207       	|  0.0016512      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2562e-06     	|  20000          	|  0.025124       	|  0.001583       	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.2495e-06     	|  20000          	|  0.02499        	|  0.0015745      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004146      	|  60             	|  0.024876       	|  0.0015674      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.2424e-06     	|  20000          	|  0.024848       	|  0.0015656      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.2052e-06     	|  20000          	|  0.024104       	|  0.0015187      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.1078e-06     	|  20000          	|  0.022157       	|  0.001396       	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  9.9046e-07     	|  20000          	|  0.019809       	|  0.0012481      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.005621       	|  2              	|  0.011242       	|  0.00070833     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0086354      	|  1              	|  0.0086354      	|  0.00054409     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00024314     	|  20             	|  0.0048627      	|  0.00030638     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.0002419      	|  20             	|  0.0048381      	|  0.00030483     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  4.7161e-05     	|  60             	|  0.0028296      	|  0.00017829     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00010194     	|  20             	|  0.0020387      	|  0.00012845     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0009089      	|  2              	|  0.0018178      	|  0.00011453     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00063145     	|  2              	|  0.0012629      	|  7.9572e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00061675     	|  2              	|  0.0012335      	|  7.7719e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.0010778      	|  1              	|  0.0010778      	|  6.7911e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00047831     	|  2              	|  0.00095661     	|  6.0273e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.0004003      	|  1              	|  0.0004003      	|  2.5221e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.6038e-06     	|  60             	|  0.00039623     	|  2.4965e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00028187     	|  1              	|  0.00028187     	|  1.776e-05      	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00026245     	|  1              	|  0.00026245     	|  1.6536e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.6593e-06     	|  60             	|  0.00021956     	|  1.3834e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.2045e-06     	|  60             	|  0.00019227     	|  1.2114e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.9658e-06     	|  60             	|  0.00017795     	|  1.1212e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.6921e-06     	|  60             	|  0.00016153     	|  1.0177e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.1754e-06     	|  20             	|  0.00012351     	|  7.7818e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  5.9729e-05     	|  2              	|  0.00011946     	|  7.5267e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.8028e-06     	|  60             	|  0.00010817     	|  6.8152e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.0055e-06     	|  20             	|  0.00010011     	|  6.3076e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  4.4537e-06     	|  22             	|  9.7982e-05     	|  6.1735e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.5522e-06     	|  60             	|  9.3133e-05     	|  5.868e-06      	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.1246e-06     	|  60             	|  6.7473e-05     	|  4.2513e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.0314e-06     	|  22             	|  6.6691e-05     	|  4.202e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.8699e-06     	|  20             	|  5.7398e-05     	|  3.6165e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.4886e-06     	|  22             	|  5.4749e-05     	|  3.4495e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1473e-06     	|  22             	|  4.724e-05      	|  2.9765e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.0858e-06     	|  22             	|  4.5887e-05     	|  2.8912e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.0682e-06     	|  22             	|  4.5501e-05     	|  2.8669e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.512e-05      	|  1              	|  4.512e-05      	|  2.8429e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.5313e-06     	|  22             	|  3.3688e-05     	|  2.1226e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.622e-05      	|  2              	|  3.2441e-05     	|  2.044e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5771e-06     	|  20             	|  3.1542e-05     	|  1.9874e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3309e-06     	|  22             	|  2.928e-05      	|  1.8448e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.3989e-06     	|  20             	|  2.7978e-05     	|  1.7628e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  1.34e-05       	|  2              	|  2.68e-05       	|  1.6886e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.199e-05      	|  2              	|  2.398e-05      	|  1.5109e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  7.2396e-06     	|  2              	|  1.4479e-05     	|  9.1229e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  7.215e-06      	|  2              	|  1.443e-05      	|  9.0918e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  8.6799e-06     	|  1              	|  8.6799e-06     	|  5.4689e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  8.1602e-06     	|  1              	|  8.1602e-06     	|  5.1415e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  7.8101e-06     	|  1              	|  7.8101e-06     	|  4.9209e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  7.6797e-06     	|  1              	|  7.6797e-06     	|  4.8387e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.7653e-06     	|  2              	|  7.5307e-06     	|  4.7448e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  7.3593e-06     	|  1              	|  7.3593e-06     	|  4.6369e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.4994e-06     	|  2              	|  6.9989e-06     	|  4.4098e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.2745e-06     	|  2              	|  6.5491e-06     	|  4.1264e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  5.8888e-06     	|  1              	|  5.8888e-06     	|  3.7103e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.875e-06      	|  2              	|  5.75e-06       	|  3.6229e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.0096e-06     	|  1              	|  5.0096e-06     	|  3.1564e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.7199e-06     	|  1              	|  4.7199e-06     	|  2.9739e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.6697e-06     	|  1              	|  4.6697e-06     	|  2.9422e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.6203e-06     	|  1              	|  4.6203e-06     	|  2.9111e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  4.6007e-06     	|  1              	|  4.6007e-06     	|  2.8988e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  4.4499e-06     	|  1              	|  4.4499e-06     	|  2.8037e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.1602e-06     	|  2              	|  4.3204e-06     	|  2.7221e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.3204e-06     	|  1              	|  4.3204e-06     	|  2.7221e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.1495e-06     	|  2              	|  4.299e-06      	|  2.7087e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.055e-06      	|  2              	|  4.1099e-06     	|  2.5895e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.9795e-06     	|  1              	|  3.9795e-06     	|  2.5074e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.9097e-06     	|  1              	|  3.9097e-06     	|  2.4634e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.9348e-06     	|  2              	|  3.8696e-06     	|  2.4381e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.8547e-06     	|  2              	|  3.7095e-06     	|  2.3372e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  3.6601e-06     	|  1              	|  3.6601e-06     	|  2.3061e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  3.6396e-06     	|  1              	|  3.6396e-06     	|  2.2932e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.6294e-06     	|  1              	|  3.6294e-06     	|  2.2867e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.58e-06       	|  1              	|  3.58e-06       	|  2.2556e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.4096e-06     	|  1              	|  3.4096e-06     	|  2.1483e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.3993e-06     	|  1              	|  3.3993e-06     	|  2.1418e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.3695e-06     	|  1              	|  3.3695e-06     	|  2.123e-07      	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.2801e-06     	|  1              	|  3.2801e-06     	|  2.0667e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.9197e-06     	|  1              	|  2.9197e-06     	|  1.8396e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.7902e-06     	|  1              	|  2.7902e-06     	|  1.758e-07      	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.5602e-06     	|  1              	|  2.5602e-06     	|  1.6131e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.2501e-06     	|  1              	|  2.2501e-06     	|  1.4177e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.2398e-06     	|  1              	|  2.2398e-06     	|  1.4112e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.2203e-06     	|  1              	|  2.2203e-06     	|  1.3989e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  2.1504e-06     	|  1              	|  2.1504e-06     	|  1.3549e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.1001e-06     	|  1              	|  2.1001e-06     	|  1.3232e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.0796e-06     	|  1              	|  2.0796e-06     	|  1.3103e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.0098e-06     	|  1              	|  2.0098e-06     	|  1.2663e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.9995e-06     	|  1              	|  1.9995e-06     	|  1.2599e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.9595e-06     	|  1              	|  1.9595e-06     	|  1.2346e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.7304e-06     	|  1              	|  1.7304e-06     	|  1.0903e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.6596e-06     	|  1              	|  1.6596e-06     	|  1.0457e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.5004e-06     	|  1              	|  1.5004e-06     	|  9.4533e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.1502e-06     	|  1              	|  1.1502e-06     	|  7.2469e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.1204e-06     	|  1              	|  1.1204e-06     	|  7.0592e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
