FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881233         	|  1518.6         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  73.695         	|  20             	|  1473.9         	|  97.059         	|
|  run_training_batch                                                                                                                                                                                         	|  0.062228       	|  20000          	|  1244.6         	|  81.957         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.061806       	|  20000          	|  1236.1         	|  81.401         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.021351       	|  20000          	|  427.02         	|  28.12          	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0064477      	|  20000          	|  128.95         	|  8.4919         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.004767       	|  20000          	|  95.339         	|  6.2783         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.4509         	|  48             	|  69.645         	|  4.5863         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  32.614         	|  1              	|  32.614         	|  2.1477         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00054576     	|  20000          	|  10.915         	|  0.71879        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.22248        	|  20             	|  4.4496         	|  0.29301        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00017921     	|  20048          	|  3.5928         	|  0.23659        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00011512     	|  20048          	|  2.308          	|  0.15199        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.7071e-05     	|  20000          	|  0.74141        	|  0.048824       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.01131        	|  48             	|  0.54287        	|  0.035749       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.1805e-05     	|  20000          	|  0.4361         	|  0.028718       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.012e-05      	|  20000          	|  0.40239        	|  0.026498       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.8673e-06     	|  20000          	|  0.097346       	|  0.0064104      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.3861e-06     	|  20000          	|  0.087722       	|  0.0057767      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.6015e-06     	|  20000          	|  0.072029       	|  0.0047433      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.035539       	|  2              	|  0.071078       	|  0.0046806      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.4443e-06     	|  20000          	|  0.068887       	|  0.0045363      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.3508e-06     	|  20000          	|  0.067017       	|  0.0044132      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.3168e-06     	|  20000          	|  0.066336       	|  0.0043684      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.0466e-06     	|  20000          	|  0.060931       	|  0.0040124      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.8099e-06     	|  20000          	|  0.056198       	|  0.0037008      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.7921e-06     	|  20000          	|  0.055842       	|  0.0036773      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.6861e-06     	|  20000          	|  0.053722       	|  0.0035377      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.6542e-06     	|  20000          	|  0.053085       	|  0.0034958      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.62e-06       	|  20000          	|  0.052399       	|  0.0034506      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  2.5369e-06     	|  20048          	|  0.05086        	|  0.0033492      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.4663e-06     	|  20000          	|  0.049326       	|  0.0032482      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.3591e-06     	|  20000          	|  0.047181       	|  0.003107       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.3148e-06     	|  20000          	|  0.046296       	|  0.0030487      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.282e-06      	|  20000          	|  0.045641       	|  0.0030055      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.2219e-06     	|  20000          	|  0.044438       	|  0.0029263      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.2074e-06     	|  20000          	|  0.044149       	|  0.0029073      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.178e-06      	|  20000          	|  0.04356        	|  0.0028685      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.1466e-06     	|  20000          	|  0.042932       	|  0.0028272      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.109e-06      	|  20000          	|  0.042181       	|  0.0027777      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.1017e-06     	|  20000          	|  0.042034       	|  0.0027681      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.020246       	|  2              	|  0.040493       	|  0.0026665      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.5421e-06     	|  20048          	|  0.030917       	|  0.0020359      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.5384e-06     	|  20000          	|  0.030768       	|  0.0020262      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.4943e-06     	|  20000          	|  0.029886       	|  0.001968       	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.4139e-06     	|  20000          	|  0.028279       	|  0.0018622      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.2805e-06     	|  20000          	|  0.025611       	|  0.0016865      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00052353     	|  48             	|  0.025129       	|  0.0016548      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2282e-06     	|  20000          	|  0.024564       	|  0.0016176      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.2009e-06     	|  20000          	|  0.024017       	|  0.0015816      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1528e-06     	|  20000          	|  0.023056       	|  0.0015183      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.1037e-06     	|  20000          	|  0.022074       	|  0.0014536      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.049e-06      	|  20000          	|  0.02098        	|  0.0013816      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0093062      	|  1              	|  0.0093062      	|  0.00061284     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00027311     	|  20             	|  0.0054623      	|  0.0003597      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00025222     	|  20             	|  0.0050444      	|  0.00033218     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.7179e-05     	|  48             	|  0.0027446      	|  0.00018074     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.8001e-05     	|  20             	|  0.00196        	|  0.00012907     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.0016671      	|  1              	|  0.0016671      	|  0.00010978     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0006846      	|  2              	|  0.0013692      	|  9.0165e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00059378     	|  2              	|  0.0011876      	|  7.8203e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00052537     	|  2              	|  0.0010507      	|  6.9194e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00036851     	|  2              	|  0.00073702     	|  4.8535e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00045927     	|  1              	|  0.00045927     	|  3.0244e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.0003611      	|  1              	|  0.0003611      	|  2.3779e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.501e-06      	|  48             	|  0.00031205     	|  2.0549e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00028552     	|  1              	|  0.00028552     	|  1.8802e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.7217e-06     	|  48             	|  0.00022664     	|  1.4925e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  4.6138e-06     	|  48             	|  0.00022146     	|  1.4584e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.2039e-06     	|  48             	|  0.00015379     	|  1.0127e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.9226e-06     	|  48             	|  0.00014028     	|  9.238e-06      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.4439e-06     	|  20             	|  0.00012888     	|  8.4869e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  6.3195e-06     	|  20             	|  0.00012639     	|  8.3231e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  0.00011425     	|  1              	|  0.00011425     	|  7.5236e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  4.9015e-06     	|  22             	|  0.00010783     	|  7.1011e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.1946e-06     	|  48             	|  0.00010534     	|  6.9369e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  4.502e-05      	|  2              	|  9.0039e-05     	|  5.9293e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.5742e-06     	|  48             	|  7.5563e-05     	|  4.976e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.0077e-06     	|  22             	|  6.617e-05      	|  4.3575e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2674e-06     	|  48             	|  6.0834e-05     	|  4.0061e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.0152e-06     	|  20             	|  6.0304e-05     	|  3.9712e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2764e-06     	|  22             	|  5.008e-05      	|  3.2979e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.2179e-06     	|  22             	|  4.8793e-05     	|  3.2131e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1677e-06     	|  22             	|  4.7689e-05     	|  3.1404e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1558e-06     	|  22             	|  4.7429e-05     	|  3.1233e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.9212e-06     	|  22             	|  4.2267e-05     	|  2.7834e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3667e-06     	|  22             	|  3.0068e-05     	|  1.98e-06       	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5014e-06     	|  20             	|  3.0029e-05     	|  1.9775e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.364e-06      	|  20             	|  2.7279e-05     	|  1.7964e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  1.1586e-05     	|  2              	|  2.3171e-05     	|  1.5259e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  9.18e-06       	|  2              	|  1.836e-05      	|  1.2091e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.47e-05       	|  1              	|  1.47e-05       	|  9.6803e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  6.5444e-06     	|  2              	|  1.3089e-05     	|  8.6193e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.07e-05       	|  1              	|  1.07e-05       	|  7.0462e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.7651e-06     	|  2              	|  9.5302e-06     	|  6.2759e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  8.7097e-06     	|  1              	|  8.7097e-06     	|  5.7355e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  3.9898e-06     	|  2              	|  7.9796e-06     	|  5.2547e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.8647e-06     	|  2              	|  5.7295e-06     	|  3.773e-07      	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.14e-06       	|  1              	|  5.14e-06       	|  3.3848e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.5453e-06     	|  2              	|  5.0906e-06     	|  3.3523e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.4e-06        	|  2              	|  4.8e-06        	|  3.1609e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  4.6901e-06     	|  1              	|  4.6901e-06     	|  3.0886e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.57e-06       	|  1              	|  4.57e-06       	|  3.0094e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.3586e-06     	|  1              	|  4.3586e-06     	|  2.8702e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  4.3204e-06     	|  1              	|  4.3204e-06     	|  2.8451e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  4.3102e-06     	|  1              	|  4.3102e-06     	|  2.8383e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.1304e-06     	|  2              	|  4.2608e-06     	|  2.8058e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.1807e-06     	|  1              	|  4.1807e-06     	|  2.7531e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.1304e-06     	|  1              	|  4.1304e-06     	|  2.72e-07       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.1295e-06     	|  1              	|  4.1295e-06     	|  2.7194e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.925e-06      	|  2              	|  3.8501e-06     	|  2.5354e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.7905e-06     	|  1              	|  3.7905e-06     	|  2.4961e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.8855e-06     	|  2              	|  3.7709e-06     	|  2.4832e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.83e-06       	|  2              	|  3.6601e-06     	|  2.4103e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.6201e-06     	|  1              	|  3.6201e-06     	|  2.3839e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.8049e-06     	|  2              	|  3.6098e-06     	|  2.3771e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.5204e-06     	|  1              	|  3.5204e-06     	|  2.3183e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.5204e-06     	|  1              	|  3.5204e-06     	|  2.3183e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.4804e-06     	|  1              	|  3.4804e-06     	|  2.2919e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.3304e-06     	|  1              	|  3.3304e-06     	|  2.1931e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  3.2997e-06     	|  1              	|  3.2997e-06     	|  2.1729e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3653e-06     	|  2              	|  2.7306e-06     	|  1.7982e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.6105e-06     	|  1              	|  2.6105e-06     	|  1.7191e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4503e-06     	|  1              	|  2.4503e-06     	|  1.6136e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.4103e-06     	|  1              	|  2.4103e-06     	|  1.5872e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.3795e-06     	|  1              	|  2.3795e-06     	|  1.567e-07      	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.3795e-06     	|  1              	|  2.3795e-06     	|  1.567e-07      	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.3395e-06     	|  1              	|  2.3395e-06     	|  1.5406e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.2696e-06     	|  1              	|  2.2696e-06     	|  1.4946e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.2305e-06     	|  1              	|  2.2305e-06     	|  1.4688e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.17e-06       	|  1              	|  2.17e-06       	|  1.429e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.1197e-06     	|  1              	|  2.1197e-06     	|  1.3959e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.0694e-06     	|  1              	|  2.0694e-06     	|  1.3627e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.3566e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.0294e-06     	|  1              	|  2.0294e-06     	|  1.3364e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  2.02e-06       	|  1              	|  2.02e-06       	|  1.3302e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.0005e-06     	|  1              	|  2.0005e-06     	|  1.3174e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.5898e-06     	|  1              	|  1.5898e-06     	|  1.0469e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.5199e-06     	|  1              	|  1.5199e-06     	|  1.0009e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.1995e-06     	|  1              	|  1.1995e-06     	|  7.8993e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.1399e-06     	|  1              	|  1.1399e-06     	|  7.5067e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0999e-06     	|  1              	|  1.0999e-06     	|  7.243e-08      	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  9.6019e-07     	|  1              	|  9.6019e-07     	|  6.3231e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
