FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  884625         	|  1821.9         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  81.809         	|  20             	|  1636.2         	|  89.806         	|
|  run_training_batch                                                                                                                                                                                         	|  0.064848       	|  20000          	|  1297.0         	|  71.188         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.064516       	|  20000          	|  1290.3         	|  70.823         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.01766        	|  20000          	|  353.21         	|  19.387         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  175.16         	|  1              	|  175.16         	|  9.614          	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.65401        	|  260            	|  170.04         	|  9.3333         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0068498      	|  20000          	|  137.0          	|  7.5194         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0021778      	|  20000          	|  43.555         	|  2.3907         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00050335     	|  20000          	|  10.067         	|  0.55256        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.000182       	|  20260          	|  3.6873         	|  0.20239        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.14869        	|  20             	|  2.9738         	|  0.16323        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00011194     	|  20260          	|  2.2679         	|  0.12448        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0031181      	|  260            	|  0.81071        	|  0.044499       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.6567e-05     	|  20000          	|  0.53133        	|  0.029164       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.105e-05      	|  20000          	|  0.421          	|  0.023108       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.9031e-05     	|  20000          	|  0.38062        	|  0.020892       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00049643     	|  260            	|  0.12907        	|  0.0070845      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  6.2629e-06     	|  20000          	|  0.12526        	|  0.0068752      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.4982e-06     	|  20000          	|  0.10996        	|  0.0060358      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  5.4299e-06     	|  20000          	|  0.1086         	|  0.0059607      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  4.9862e-06     	|  20000          	|  0.099725       	|  0.0054737      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.7259e-06     	|  20000          	|  0.094519       	|  0.005188       	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  4.6885e-06     	|  20000          	|  0.09377        	|  0.0051469      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  4.4866e-06     	|  20000          	|  0.089732       	|  0.0049252      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  4.4072e-06     	|  20000          	|  0.088144       	|  0.004838       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  4.3733e-06     	|  20000          	|  0.087466       	|  0.0048008      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.043148       	|  2              	|  0.086295       	|  0.0047366      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  4.2936e-06     	|  20000          	|  0.085873       	|  0.0047134      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  4.2713e-06     	|  20000          	|  0.085426       	|  0.0046889      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  4.1798e-06     	|  20000          	|  0.083595       	|  0.0045884      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  4.1432e-06     	|  20000          	|  0.082864       	|  0.0045483      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  4.1251e-06     	|  20000          	|  0.082501       	|  0.0045283      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  4.0353e-06     	|  20000          	|  0.080707       	|  0.0044298      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  4.0169e-06     	|  20000          	|  0.080337       	|  0.0044096      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  4.0028e-06     	|  20000          	|  0.080056       	|  0.0043941      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  3.9991e-06     	|  20000          	|  0.079981       	|  0.00439        	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  3.9936e-06     	|  20000          	|  0.079871       	|  0.004384       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  3.9847e-06     	|  20000          	|  0.079693       	|  0.0043742      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  3.9613e-06     	|  20000          	|  0.079227       	|  0.0043486      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  3.9584e-06     	|  20000          	|  0.079167       	|  0.0043453      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.4828e-06     	|  20260          	|  0.070562       	|  0.003873       	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  3.0277e-06     	|  20000          	|  0.060553       	|  0.0033236      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  2.8936e-06     	|  20260          	|  0.058625       	|  0.0032178      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  2.6259e-06     	|  20000          	|  0.052518       	|  0.0028826      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  2.5893e-06     	|  20000          	|  0.051786       	|  0.0028425      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  2.5843e-06     	|  20000          	|  0.051686       	|  0.002837       	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  2.4807e-06     	|  20000          	|  0.049615       	|  0.0027233      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  2.4071e-06     	|  20000          	|  0.048141       	|  0.0026424      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  2.3803e-06     	|  20000          	|  0.047606       	|  0.002613       	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  2.3255e-06     	|  20000          	|  0.04651        	|  0.0025528      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  2.3195e-06     	|  20000          	|  0.046391       	|  0.0025463      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0066657      	|  2              	|  0.013331       	|  0.00073174     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00040713     	|  20             	|  0.0081427      	|  0.00044694     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00030638     	|  20             	|  0.0061275      	|  0.00033633     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0049341      	|  1              	|  0.0049341      	|  0.00027082     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  1.7367e-05     	|  260            	|  0.0045153      	|  0.00024784     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00012436     	|  20             	|  0.0024872      	|  0.00013652     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  8.4585e-06     	|  260            	|  0.0021992      	|  0.00012071     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0010915      	|  2              	|  0.0021829      	|  0.00011982     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  5.1746e-06     	|  260            	|  0.0013454      	|  7.3847e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  5.1658e-06     	|  260            	|  0.0013431      	|  7.372e-05      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  4.9951e-06     	|  260            	|  0.0012987      	|  7.1285e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  4.7703e-06     	|  260            	|  0.0012403      	|  6.8077e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00061638     	|  2              	|  0.0012328      	|  6.7663e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00055017     	|  2              	|  0.0011003      	|  6.0395e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  4.0846e-06     	|  260            	|  0.001062       	|  5.8291e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.0010352      	|  1              	|  0.0010352      	|  5.6819e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  3.5122e-06     	|  260            	|  0.00091318     	|  5.0123e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  2.9639e-06     	|  260            	|  0.00077062     	|  4.2298e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00032406     	|  2              	|  0.00064812     	|  3.5574e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00061837     	|  1              	|  0.00061837     	|  3.3941e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00060984     	|  1              	|  0.00060984     	|  3.3473e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00041164     	|  1              	|  0.00041164     	|  2.2594e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  7.5498e-06     	|  20             	|  0.000151       	|  8.2879e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  6.8548e-06     	|  20             	|  0.0001371      	|  7.525e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  5.3618e-06     	|  22             	|  0.00011796     	|  6.4746e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  5.019e-06      	|  22             	|  0.00011042     	|  6.0606e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  5.1822e-05     	|  2              	|  0.00010364     	|  5.6888e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  4.1524e-06     	|  22             	|  9.1353e-05     	|  5.0142e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  4.0984e-06     	|  22             	|  9.0165e-05     	|  4.949e-06      	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  3.9997e-06     	|  22             	|  8.7994e-05     	|  4.8298e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  3.9808e-06     	|  22             	|  8.7578e-05     	|  4.807e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  4.3092e-06     	|  20             	|  8.6185e-05     	|  4.7305e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  3.4984e-06     	|  22             	|  7.6964e-05     	|  4.2244e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  3.1269e-06     	|  22             	|  6.8791e-05     	|  3.7758e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  3.279e-06      	|  20             	|  6.5579e-05     	|  3.5995e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  3.0382e-06     	|  20             	|  6.0763e-05     	|  3.3352e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  5.727e-05      	|  1              	|  5.727e-05      	|  3.1434e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.2083e-05     	|  2              	|  2.4165e-05     	|  1.3264e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  9.9516e-06     	|  2              	|  1.9903e-05     	|  1.0925e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  9.7081e-06     	|  2              	|  1.9416e-05     	|  1.0657e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  8.7298e-06     	|  2              	|  1.746e-05      	|  9.5832e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.4527e-05     	|  1              	|  1.4527e-05     	|  7.9735e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.4109e-05     	|  1              	|  1.4109e-05     	|  7.744e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.8664e-06     	|  2              	|  1.1733e-05     	|  6.4399e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  5.6573e-06     	|  2              	|  1.1315e-05     	|  6.2104e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.0895e-05     	|  1              	|  1.0895e-05     	|  5.9799e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  1.0407e-05     	|  1              	|  1.0407e-05     	|  5.712e-07      	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  9.9177e-06     	|  1              	|  9.9177e-06     	|  5.4436e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  4.9584e-06     	|  2              	|  9.9167e-06     	|  5.4431e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  9.0105e-06     	|  1              	|  9.0105e-06     	|  4.9457e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  8.5197e-06     	|  1              	|  8.5197e-06     	|  4.6763e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  8.5197e-06     	|  1              	|  8.5197e-06     	|  4.6763e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  8.1016e-06     	|  1              	|  8.1016e-06     	|  4.4468e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  8.1016e-06     	|  1              	|  8.1016e-06     	|  4.4468e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  4.0503e-06     	|  2              	|  8.1006e-06     	|  4.4463e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  8.0308e-06     	|  1              	|  8.0308e-06     	|  4.408e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  3.8063e-06     	|  2              	|  7.6126e-06     	|  4.1784e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  3.7719e-06     	|  2              	|  7.5437e-06     	|  4.1406e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.7714e-06     	|  2              	|  7.5428e-06     	|  4.1401e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  7.5419e-06     	|  1              	|  7.5419e-06     	|  4.1396e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  3.7369e-06     	|  2              	|  7.4739e-06     	|  4.1023e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  3.5968e-06     	|  2              	|  7.1935e-06     	|  3.9484e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  7.1237e-06     	|  1              	|  7.1237e-06     	|  3.9101e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  3.5618e-06     	|  2              	|  7.1237e-06     	|  3.9101e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  6.5649e-06     	|  1              	|  6.5649e-06     	|  3.6033e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  6.2166e-06     	|  1              	|  6.2166e-06     	|  3.4122e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  6.2166e-06     	|  1              	|  6.2166e-06     	|  3.4122e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  6.2156e-06     	|  1              	|  6.2156e-06     	|  3.4117e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  6.2147e-06     	|  1              	|  6.2147e-06     	|  3.4111e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  6.1467e-06     	|  1              	|  6.1467e-06     	|  3.3738e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  3.3733e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  3.1049e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  4.3297e-06     	|  1              	|  4.3297e-06     	|  2.3765e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  4.2608e-06     	|  1              	|  4.2608e-06     	|  2.3387e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  4.191e-06      	|  1              	|  4.191e-06      	|  2.3003e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  4.19e-06       	|  1              	|  4.19e-06       	|  2.2998e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.19e-06       	|  1              	|  4.19e-06       	|  2.2998e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  2.1086e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  2.1086e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  2.1086e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.8408e-06     	|  1              	|  3.8408e-06     	|  2.1081e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.8408e-06     	|  1              	|  3.8408e-06     	|  2.1081e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.8408e-06     	|  1              	|  3.8408e-06     	|  2.1081e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  2.0703e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  2.0698e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  2.0698e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  3.702e-06      	|  1              	|  3.702e-06      	|  2.032e-07      	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.7001e-06     	|  1              	|  3.7001e-06     	|  2.0309e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.3518e-06     	|  1              	|  3.3518e-06     	|  1.8398e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  3.282e-06      	|  1              	|  3.282e-06      	|  1.8014e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.3041e-06     	|  1              	|  2.3041e-06     	|  1.2647e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
