FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  888241         	|  2196.0         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  94.498         	|  20             	|  1890.0         	|  86.063         	|
|  run_training_batch                                                                                                                                                                                         	|  0.071931       	|  20000          	|  1438.6         	|  65.51          	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.071604       	|  20000          	|  1432.1         	|  65.212         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.019199       	|  20000          	|  383.97         	|  17.485         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.60687        	|  486            	|  294.94         	|  13.431         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  293.86         	|  1              	|  293.86         	|  13.381         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0061363      	|  20000          	|  122.73         	|  5.5886         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0021533      	|  20000          	|  43.066         	|  1.9611         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00049727     	|  20000          	|  9.9454         	|  0.45288        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.2131         	|  20             	|  4.262          	|  0.19408        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00018576     	|  20486          	|  3.8054         	|  0.17329        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00011781     	|  20486          	|  2.4134         	|  0.1099         	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.003477       	|  486            	|  1.6898         	|  0.076949       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.5789e-05     	|  20000          	|  0.51577        	|  0.023487       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.1307e-05     	|  20000          	|  0.42615        	|  0.019405       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.8877e-05     	|  20000          	|  0.37753        	|  0.017192       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004765      	|  486            	|  0.23158        	|  0.010545       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  6.3532e-06     	|  20000          	|  0.12706        	|  0.0057861      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  5.5176e-06     	|  20000          	|  0.11035        	|  0.0050251      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.2254e-06     	|  20000          	|  0.10451        	|  0.004759       	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  4.9913e-06     	|  20000          	|  0.099826       	|  0.0045458      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.049295       	|  2              	|  0.098589       	|  0.0044894      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  4.6921e-06     	|  20000          	|  0.093842       	|  0.0042733      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  4.4575e-06     	|  20000          	|  0.089151       	|  0.0040596      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  4.3986e-06     	|  20000          	|  0.087973       	|  0.004006       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.3911e-06     	|  20000          	|  0.087822       	|  0.0039991      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  4.3301e-06     	|  20000          	|  0.086601       	|  0.0039435      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  4.2963e-06     	|  20000          	|  0.085927       	|  0.0039128      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  4.2767e-06     	|  20000          	|  0.085535       	|  0.003895       	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  4.21e-06       	|  20000          	|  0.084199       	|  0.0038342      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  4.114e-06      	|  20000          	|  0.082281       	|  0.0037468      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  4.1035e-06     	|  20000          	|  0.08207        	|  0.0037372      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  4.0916e-06     	|  20000          	|  0.081832       	|  0.0037264      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  4.0162e-06     	|  20000          	|  0.080325       	|  0.0036577      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  3.9906e-06     	|  20000          	|  0.079812       	|  0.0036344      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  3.9771e-06     	|  20000          	|  0.079543       	|  0.0036221      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  3.961e-06      	|  20000          	|  0.079221       	|  0.0036075      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  3.9563e-06     	|  20000          	|  0.079127       	|  0.0036032      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  3.9465e-06     	|  20000          	|  0.078929       	|  0.0035942      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  3.9212e-06     	|  20000          	|  0.078425       	|  0.0035712      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.5451e-06     	|  20486          	|  0.072625       	|  0.0033071      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  2.743e-06      	|  20000          	|  0.054859       	|  0.0024981      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  2.6589e-06     	|  20486          	|  0.05447        	|  0.0024804      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  2.669e-06      	|  20000          	|  0.05338        	|  0.0024307      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  2.4912e-06     	|  20000          	|  0.049824       	|  0.0022688      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  2.4187e-06     	|  20000          	|  0.048374       	|  0.0022028      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  2.4036e-06     	|  20000          	|  0.048072       	|  0.002189       	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  2.3968e-06     	|  20000          	|  0.047935       	|  0.0021828      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  2.3472e-06     	|  20000          	|  0.046944       	|  0.0021377      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  2.3199e-06     	|  20000          	|  0.046397       	|  0.0021128      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  2.2661e-06     	|  20000          	|  0.045322       	|  0.0020638      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0067986      	|  2              	|  0.013597       	|  0.00061917     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  1.2455e-05     	|  486            	|  0.006053       	|  0.00027563     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00026148     	|  20             	|  0.0052296      	|  0.00023814     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00025288     	|  20             	|  0.0050576      	|  0.0002303      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0047112      	|  1              	|  0.0047112      	|  0.00021453     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  8.2684e-06     	|  486            	|  0.0040185      	|  0.00018299     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  5.1354e-06     	|  486            	|  0.0024958      	|  0.00011365     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0012152      	|  2              	|  0.0024304      	|  0.00011067     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  4.8304e-06     	|  486            	|  0.0023476      	|  0.0001069      	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  4.8171e-06     	|  486            	|  0.0023411      	|  0.00010661     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011651     	|  20             	|  0.0023302      	|  0.00010611     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  4.7229e-06     	|  486            	|  0.0022953      	|  0.00010452     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  3.8806e-06     	|  486            	|  0.001886       	|  8.5881e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  3.27e-06       	|  486            	|  0.0015892      	|  7.2368e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  2.9836e-06     	|  486            	|  0.00145        	|  6.6029e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00070821     	|  2              	|  0.0014164      	|  6.45e-05       	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00056703     	|  2              	|  0.0011341      	|  5.1642e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00042348     	|  2              	|  0.00084695     	|  3.8567e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00074324     	|  1              	|  0.00074324     	|  3.3845e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00049531     	|  1              	|  0.00049531     	|  2.2555e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00047778     	|  1              	|  0.00047778     	|  2.1756e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00043196     	|  1              	|  0.00043196     	|  1.967e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  7.6649e-06     	|  20             	|  0.0001533      	|  6.9807e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  7.2391e-06     	|  20             	|  0.00014478     	|  6.5929e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  6.8164e-05     	|  2              	|  0.00013633     	|  6.208e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  4.5998e-06     	|  22             	|  0.0001012      	|  4.6081e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  4.5967e-06     	|  22             	|  0.00010113     	|  4.605e-06      	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  4.2032e-06     	|  22             	|  9.2471e-05     	|  4.2108e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.613e-05      	|  2              	|  9.226e-05      	|  4.2012e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  4.0507e-06     	|  22             	|  8.9115e-05     	|  4.058e-06      	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  4.0411e-06     	|  22             	|  8.8905e-05     	|  4.0484e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  3.968e-06      	|  22             	|  8.7297e-05     	|  3.9752e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  4.2322e-06     	|  20             	|  8.4644e-05     	|  3.8544e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  3.0095e-06     	|  22             	|  6.621e-05      	|  3.015e-06      	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  2.8856e-06     	|  22             	|  6.3484e-05     	|  2.8908e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  2.8916e-06     	|  20             	|  5.7832e-05     	|  2.6335e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  2.5351e-06     	|  20             	|  5.0702e-05     	|  2.3088e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.4977e-05     	|  1              	|  4.4977e-05     	|  2.0481e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.6098e-05     	|  2              	|  3.2197e-05     	|  1.4661e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.086e-05      	|  2              	|  2.172e-05      	|  9.8907e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  1.0616e-05     	|  2              	|  2.1232e-05     	|  9.6685e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  9.7081e-06     	|  2              	|  1.9416e-05     	|  8.8415e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  5.9018e-06     	|  2              	|  1.1804e-05     	|  5.375e-07      	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  1.1734e-05     	|  1              	|  1.1734e-05     	|  5.3432e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  1.1733e-05     	|  1              	|  1.1733e-05     	|  5.3427e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.1315e-05     	|  1              	|  1.1315e-05     	|  5.1523e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  1.0964e-05     	|  1              	|  1.0964e-05     	|  4.9924e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  1.0895e-05     	|  1              	|  1.0895e-05     	|  4.9611e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  4.9933e-06     	|  2              	|  9.9866e-06     	|  4.5476e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  4.9588e-06     	|  2              	|  9.9177e-06     	|  4.5162e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  4.9584e-06     	|  2              	|  9.9167e-06     	|  4.5158e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  9.4986e-06     	|  1              	|  9.4986e-06     	|  4.3253e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  9.0105e-06     	|  1              	|  9.0105e-06     	|  4.1031e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  9.0087e-06     	|  1              	|  9.0087e-06     	|  4.1023e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  4.2948e-06     	|  2              	|  8.5896e-06     	|  3.9114e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  4.0508e-06     	|  2              	|  8.1016e-06     	|  3.6892e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  8.0317e-06     	|  1              	|  8.0317e-06     	|  3.6574e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  4.0159e-06     	|  2              	|  8.0317e-06     	|  3.6574e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  7.6117e-06     	|  1              	|  7.6117e-06     	|  3.4661e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  3.8058e-06     	|  2              	|  7.6117e-06     	|  3.4661e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  7.5437e-06     	|  1              	|  7.5437e-06     	|  3.4352e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  7.5437e-06     	|  1              	|  7.5437e-06     	|  3.4352e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  7.5428e-06     	|  1              	|  7.5428e-06     	|  3.4347e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  7.5428e-06     	|  1              	|  7.5428e-06     	|  3.4347e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  3.5963e-06     	|  2              	|  7.1926e-06     	|  3.2753e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  6.6347e-06     	|  1              	|  6.6347e-06     	|  3.0212e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  2.7986e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  2.7986e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  6.0759e-06     	|  1              	|  6.0759e-06     	|  2.7668e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  5.7276e-06     	|  1              	|  5.7276e-06     	|  2.6082e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  5.7267e-06     	|  1              	|  5.7267e-06     	|  2.6078e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  5.6578e-06     	|  1              	|  5.6578e-06     	|  2.5764e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  2.5759e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  4.19e-06       	|  1              	|  4.19e-06       	|  1.908e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  1.7494e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  1.7494e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.8408e-06     	|  1              	|  3.8408e-06     	|  1.749e-07      	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.7176e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.7176e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.7176e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.7176e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.7172e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.7172e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.7172e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.7011e-06     	|  1              	|  3.7011e-06     	|  1.6853e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.3528e-06     	|  1              	|  3.3528e-06     	|  1.5267e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  3.2829e-06     	|  1              	|  3.2829e-06     	|  1.4949e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.282e-06      	|  1              	|  3.282e-06      	|  1.4945e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.282e-06      	|  1              	|  3.282e-06      	|  1.4945e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  2.305e-06      	|  1              	|  2.305e-06      	|  1.0496e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.9558e-06     	|  1              	|  1.9558e-06     	|  8.906e-08      	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
