FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881617         	|  1035.3         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  50.246         	|  20             	|  1004.9         	|  97.068         	|
|  run_training_batch                                                                                                                                                                                         	|  0.043248       	|  20000          	|  864.97         	|  83.549         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.042985       	|  20000          	|  859.7          	|  83.04          	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.011944       	|  20000          	|  238.88         	|  23.074         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0035229      	|  20000          	|  70.458         	|  6.8057         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.63972        	|  72             	|  46.06          	|  4.449          	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0019462      	|  20000          	|  38.924         	|  3.7598         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  16.827         	|  1              	|  16.827         	|  1.6254         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00045162     	|  20000          	|  9.0323         	|  0.87245        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.17249        	|  20             	|  3.4498         	|  0.33323        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00012733     	|  20072          	|  2.5557         	|  0.24686        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  8.1778e-05     	|  20072          	|  1.6414         	|  0.15855        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0075822      	|  72             	|  0.54592        	|  0.052732       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.0023e-05     	|  20000          	|  0.40047        	|  0.038682       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.5138e-05     	|  20000          	|  0.30277        	|  0.029245       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.5083e-05     	|  20000          	|  0.30167        	|  0.029138       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.5128e-06     	|  20000          	|  0.090255       	|  0.0087179      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.042703       	|  2              	|  0.085407       	|  0.0082496      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.4196e-06     	|  20000          	|  0.068393       	|  0.0066062      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.1409e-06     	|  20000          	|  0.062817       	|  0.0060676      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.1225e-06     	|  20000          	|  0.062449       	|  0.0060321      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.9581e-06     	|  20000          	|  0.059163       	|  0.0057147      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.8569e-06     	|  20000          	|  0.057138       	|  0.005519       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.6753e-06     	|  20000          	|  0.053506       	|  0.0051682      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.6153e-06     	|  20000          	|  0.052306       	|  0.0050524      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.4373e-06     	|  20000          	|  0.048746       	|  0.0047085      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.3911e-06     	|  20000          	|  0.047823       	|  0.0046193      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.3518e-06     	|  20000          	|  0.047036       	|  0.0045433      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.3352e-06     	|  20000          	|  0.046705       	|  0.0045113      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.2767e-06     	|  20000          	|  0.045534       	|  0.0043983      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.2473e-06     	|  20000          	|  0.044946       	|  0.0043414      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2229e-06     	|  20000          	|  0.044458       	|  0.0042943      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.179e-06      	|  20000          	|  0.043579       	|  0.0042094      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1494e-06     	|  20000          	|  0.042987       	|  0.0041522      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.1474e-06     	|  20000          	|  0.042948       	|  0.0041485      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0945e-06     	|  20000          	|  0.041889       	|  0.0040461      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0794e-06     	|  20000          	|  0.041588       	|  0.0040171      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.0599e-06     	|  20000          	|  0.041198       	|  0.0039794      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.9625e-06     	|  20000          	|  0.03925        	|  0.0037913      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00039819     	|  72             	|  0.02867        	|  0.0027693      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.3047e-06     	|  20072          	|  0.026187       	|  0.0025295      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.3022e-06     	|  20072          	|  0.026137       	|  0.0025246      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.1399e-06     	|  20000          	|  0.022797       	|  0.002202       	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.0611e-06     	|  20000          	|  0.021221       	|  0.0020498      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.0195e-06     	|  20000          	|  0.020389       	|  0.0019694      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  9.9451e-07     	|  20000          	|  0.01989        	|  0.0019212      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  9.527e-07      	|  20000          	|  0.019054       	|  0.0018405      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  9.3727e-07     	|  20000          	|  0.018745       	|  0.0018107      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  9.0766e-07     	|  20000          	|  0.018153       	|  0.0017535      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  8.5639e-07     	|  20000          	|  0.017128       	|  0.0016544      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.5503e-07     	|  20000          	|  0.017101       	|  0.0016518      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0062479      	|  2              	|  0.012496       	|  0.001207       	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00025995     	|  20             	|  0.005199       	|  0.00050218     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00022127     	|  20             	|  0.0044254      	|  0.00042745     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  3.8224e-05     	|  72             	|  0.0027522      	|  0.00026584     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0020311      	|  1              	|  0.0020311      	|  0.00019619     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.238e-05      	|  20             	|  0.0018476      	|  0.00017846     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00064019     	|  2              	|  0.0012804      	|  0.00012368     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00056933     	|  2              	|  0.0011387      	|  0.00010999     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00044104     	|  2              	|  0.00088208     	|  8.5202e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00054479     	|  1              	|  0.00054479     	|  5.2622e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00025615     	|  2              	|  0.0005123      	|  4.9485e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  5.9383e-06     	|  72             	|  0.00042756     	|  4.1298e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00031922     	|  1              	|  0.00031922     	|  3.0834e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00026539     	|  1              	|  0.00026539     	|  2.5634e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.0002474      	|  1              	|  0.0002474      	|  2.3897e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.1405e-06     	|  72             	|  0.00022612     	|  2.1841e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.7157e-06     	|  72             	|  0.00019553     	|  1.8887e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.6866e-06     	|  72             	|  0.00019343     	|  1.8684e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.49e-06       	|  72             	|  0.00017928     	|  1.7317e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.4683e-06     	|  72             	|  0.00017772     	|  1.7166e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.524e-06      	|  20             	|  0.00011048     	|  1.0671e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.4702e-06     	|  72             	|  0.00010585     	|  1.0225e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.6148e-06     	|  20             	|  9.2296e-05     	|  8.915e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  4.4515e-05     	|  2              	|  8.903e-05      	|  8.5996e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.149e-06      	|  72             	|  8.2729e-05     	|  7.991e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.0955e-06     	|  22             	|  6.8101e-05     	|  6.578e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.9171e-06     	|  22             	|  6.4177e-05     	|  6.1989e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.624e-06      	|  20             	|  5.248e-05      	|  5.0692e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1969e-06     	|  22             	|  4.8331e-05     	|  4.6684e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1608e-06     	|  22             	|  4.7538e-05     	|  4.5918e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.0174e-06     	|  22             	|  4.4382e-05     	|  4.287e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  1.9868e-06     	|  22             	|  4.3709e-05     	|  4.2219e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.3079e-05     	|  1              	|  4.3079e-05     	|  4.1611e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5274e-06     	|  20             	|  3.0547e-05     	|  2.9506e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3758e-06     	|  22             	|  3.0268e-05     	|  2.9236e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.4714e-06     	|  20             	|  2.9428e-05     	|  2.8425e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.291e-06      	|  22             	|  2.8402e-05     	|  2.7434e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  7.3649e-06     	|  2              	|  1.473e-05      	|  1.4228e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  6.3851e-06     	|  2              	|  1.277e-05      	|  1.2335e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  5.7202e-06     	|  2              	|  1.144e-05      	|  1.105e-06      	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.7702e-06     	|  2              	|  9.5405e-06     	|  9.2153e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  9.4799e-06     	|  1              	|  9.4799e-06     	|  9.1569e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  4.6552e-06     	|  2              	|  9.3104e-06     	|  8.9931e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  8.5598e-06     	|  1              	|  8.5598e-06     	|  8.2681e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.7448e-06     	|  2              	|  7.4897e-06     	|  7.2344e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.4696e-06     	|  2              	|  6.9393e-06     	|  6.7028e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  6.2706e-06     	|  1              	|  6.2706e-06     	|  6.0569e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  5.3998e-06     	|  1              	|  5.3998e-06     	|  5.2158e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.1102e-06     	|  1              	|  5.1102e-06     	|  4.936e-07      	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.5453e-06     	|  2              	|  5.0906e-06     	|  4.9171e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.3497e-06     	|  2              	|  4.6995e-06     	|  4.5393e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.1299e-06     	|  2              	|  4.2599e-06     	|  4.1147e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.0494e-06     	|  1              	|  4.0494e-06     	|  3.9114e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  3.9097e-06     	|  1              	|  3.9097e-06     	|  3.7765e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  3.8696e-06     	|  1              	|  3.8696e-06     	|  3.7378e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.8748e-06     	|  2              	|  3.7495e-06     	|  3.6217e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.8599e-06     	|  2              	|  3.7197e-06     	|  3.5929e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  3.6303e-06     	|  1              	|  3.6303e-06     	|  3.5066e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.4701e-06     	|  1              	|  3.4701e-06     	|  3.3518e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.4105e-06     	|  1              	|  3.4105e-06     	|  3.2943e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.6503e-06     	|  2              	|  3.3006e-06     	|  3.1881e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.6354e-06     	|  2              	|  3.2708e-06     	|  3.1593e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.1404e-06     	|  1              	|  3.1404e-06     	|  3.0334e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.9998e-06     	|  1              	|  2.9998e-06     	|  2.8976e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.9095e-06     	|  1              	|  2.9095e-06     	|  2.8103e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.7409e-06     	|  1              	|  2.7409e-06     	|  2.6475e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.6496e-06     	|  1              	|  2.6496e-06     	|  2.5593e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.6003e-06     	|  1              	|  2.6003e-06     	|  2.5116e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.4596e-06     	|  1              	|  2.4596e-06     	|  2.3758e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.3898e-06     	|  1              	|  2.3898e-06     	|  2.3083e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.36e-06       	|  1              	|  2.36e-06       	|  2.2795e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.3097e-06     	|  1              	|  2.3097e-06     	|  2.231e-07      	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.21e-06       	|  1              	|  2.21e-06       	|  2.1347e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.1895e-06     	|  1              	|  2.1895e-06     	|  2.1149e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  2.1793e-06     	|  1              	|  2.1793e-06     	|  2.105e-07      	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  2.0672e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.1392e-06     	|  1              	|  2.1392e-06     	|  2.0663e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1392e-06     	|  1              	|  2.1392e-06     	|  2.0663e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1001e-06     	|  1              	|  2.1001e-06     	|  2.0286e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.0796e-06     	|  1              	|  2.0796e-06     	|  2.0088e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.9899e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.9899e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  1.9697e-06     	|  1              	|  1.9697e-06     	|  1.9026e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.9399e-06     	|  1              	|  1.9399e-06     	|  1.8738e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.9204e-06     	|  1              	|  1.9204e-06     	|  1.8549e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.7099e-06     	|  1              	|  1.7099e-06     	|  1.6516e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.4799e-06     	|  1              	|  1.4799e-06     	|  1.4294e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.3802e-06     	|  1              	|  1.3802e-06     	|  1.3332e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.2498e-06     	|  1              	|  1.2498e-06     	|  1.2072e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1595e-06     	|  1              	|  1.1595e-06     	|  1.12e-07       	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  9.2946e-07     	|  1              	|  9.2946e-07     	|  8.9778e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
