FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881649         	|  619.41         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  28.039         	|  20             	|  560.78         	|  90.534         	|
|  run_training_batch                                                                                                                                                                                         	|  0.023515       	|  20000          	|  470.3          	|  75.926         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.023279       	|  20000          	|  465.57         	|  75.163         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.0088692      	|  20000          	|  177.38         	|  28.637         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  45.634         	|  1              	|  45.634         	|  7.3672         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.001923       	|  20000          	|  38.46          	|  6.2091         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0019007      	|  20000          	|  38.014         	|  6.1371         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.39865        	|  74             	|  29.5           	|  4.7626         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00046813     	|  20000          	|  9.3626         	|  1.5115         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.17993        	|  20             	|  3.5985         	|  0.58096        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.0001157      	|  20074          	|  2.3226         	|  0.37496        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.3116e-05     	|  20074          	|  1.4677         	|  0.23696        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0049142      	|  74             	|  0.36365        	|  0.058709       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.7507e-05     	|  20000          	|  0.35013        	|  0.056526       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.3668e-05     	|  20000          	|  0.27336        	|  0.044131       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.2765e-05     	|  20000          	|  0.2553         	|  0.041217       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.6303e-06     	|  20000          	|  0.092606       	|  0.014951       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.044279       	|  2              	|  0.088558       	|  0.014297       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.3024e-06     	|  20000          	|  0.066049       	|  0.010663       	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.2288e-06     	|  20000          	|  0.064575       	|  0.010425       	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.0183e-06     	|  20000          	|  0.060366       	|  0.0097456      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.8187e-06     	|  20000          	|  0.056374       	|  0.0091012      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.7197e-06     	|  20000          	|  0.054393       	|  0.0087814      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.5523e-06     	|  20000          	|  0.051046       	|  0.008241       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.5331e-06     	|  20000          	|  0.050663       	|  0.0081792      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.3828e-06     	|  20000          	|  0.047655       	|  0.0076936      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.358e-06      	|  20000          	|  0.047159       	|  0.0076135      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3074e-06     	|  20000          	|  0.046148       	|  0.0074502      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2671e-06     	|  20000          	|  0.045341       	|  0.00732        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2341e-06     	|  20000          	|  0.044682       	|  0.0072137      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2321e-06     	|  20000          	|  0.044642       	|  0.0072072      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1735e-06     	|  20000          	|  0.043469       	|  0.0070178      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0704e-06     	|  20000          	|  0.041408       	|  0.0066851      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.0498e-06     	|  20000          	|  0.040997       	|  0.0066186      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0336e-06     	|  20000          	|  0.040671       	|  0.0065661      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.0169e-06     	|  20000          	|  0.040339       	|  0.0065124      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  1.9874e-06     	|  20000          	|  0.039748       	|  0.006417       	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9452e-06     	|  20000          	|  0.038905       	|  0.0062809      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9338e-06     	|  20000          	|  0.038676       	|  0.006244       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00049329     	|  74             	|  0.036503       	|  0.0058932      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.1207e-06     	|  20074          	|  0.022497       	|  0.003632       	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.0804e-06     	|  20074          	|  0.021688       	|  0.0035014      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.0442e-06     	|  20000          	|  0.020885       	|  0.0033717      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.006e-06      	|  20000          	|  0.020121       	|  0.0032483      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.3597e-07     	|  20000          	|  0.018719       	|  0.0030221      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.9197e-07     	|  20000          	|  0.017839       	|  0.0028801      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.3226e-07     	|  20000          	|  0.016645       	|  0.0026873      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  7.9412e-07     	|  20000          	|  0.015882       	|  0.0025641      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  7.8032e-07     	|  20000          	|  0.015606       	|  0.0025195      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  7.6325e-07     	|  20000          	|  0.015265       	|  0.0024644      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  7.5199e-07     	|  20000          	|  0.01504        	|  0.0024281      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0058024      	|  2              	|  0.011605       	|  0.0018735      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00023847     	|  20             	|  0.0047693      	|  0.00076997     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00022429     	|  20             	|  0.0044857      	|  0.00072419     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0034101      	|  1              	|  0.0034101      	|  0.00055054     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  4.1049e-05     	|  74             	|  0.0030377      	|  0.00049041     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.4607e-05     	|  20             	|  0.0018921      	|  0.00030547     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00051103     	|  2              	|  0.0010221      	|  0.000165       	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00044909     	|  2              	|  0.00089817     	|  0.000145       	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00041741     	|  2              	|  0.00083481     	|  0.00013477     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00030124     	|  2              	|  0.00060247     	|  9.7265e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.5503e-06     	|  74             	|  0.00055872     	|  9.0202e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00043079     	|  1              	|  0.00043079     	|  6.9547e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.48e-06       	|  74             	|  0.00033152     	|  5.3521e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00030654     	|  1              	|  0.00030654     	|  4.9488e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00026485     	|  1              	|  0.00026485     	|  4.2758e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00025866     	|  1              	|  0.00025866     	|  4.1758e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.3284e-06     	|  74             	|  0.0002463      	|  3.9763e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  3.2235e-06     	|  74             	|  0.00023854     	|  3.851e-05      	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.0145e-06     	|  74             	|  0.00022307     	|  3.6014e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  9.3393e-05     	|  2              	|  0.00018679     	|  3.0155e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.9288e-06     	|  74             	|  0.00014273     	|  2.3043e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7436e-06     	|  74             	|  0.00012902     	|  2.083e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.72e-06       	|  20             	|  0.0001144      	|  1.8469e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  5.3956e-06     	|  20             	|  0.00010791     	|  1.7422e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.4015e-06     	|  74             	|  0.00010371     	|  1.6743e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.9375e-06     	|  20             	|  9.875e-05      	|  1.5942e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.855e-06      	|  22             	|  8.4811e-05     	|  1.3692e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.0269e-06     	|  22             	|  6.6591e-05     	|  1.0751e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  5.649e-05      	|  1              	|  5.649e-05      	|  9.12e-06       	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  2.4375e-05     	|  2              	|  4.8749e-05     	|  7.8702e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1909e-06     	|  22             	|  4.8201e-05     	|  7.7816e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  2.379e-05      	|  2              	|  4.7579e-05     	|  7.6814e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1027e-06     	|  22             	|  4.6259e-05     	|  7.4681e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  1.9972e-06     	|  22             	|  4.3938e-05     	|  7.0935e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  1.94e-06       	|  22             	|  4.2681e-05     	|  6.8905e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3182e-06     	|  22             	|  2.9e-05        	|  4.6819e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3058e-06     	|  22             	|  2.8729e-05     	|  4.638e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.418e-06      	|  20             	|  2.8361e-05     	|  4.5786e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.2171e-06     	|  20             	|  2.4342e-05     	|  3.9298e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.567e-05      	|  1              	|  1.567e-05      	|  2.5297e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.468e-05      	|  1              	|  1.468e-05      	|  2.3699e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.4706e-06     	|  2              	|  1.0941e-05     	|  1.7664e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.2154e-06     	|  2              	|  1.0431e-05     	|  1.684e-06      	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  8.5998e-06     	|  1              	|  8.5998e-06     	|  1.3884e-06     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  8.57e-06       	|  1              	|  8.57e-06       	|  1.3836e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  4.1747e-06     	|  2              	|  8.3493e-06     	|  1.3479e-06     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  8.1304e-06     	|  1              	|  8.1304e-06     	|  1.3126e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.7798e-06     	|  2              	|  7.5595e-06     	|  1.2204e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.9197e-06     	|  2              	|  5.8394e-06     	|  9.4273e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.7404e-06     	|  2              	|  5.4808e-06     	|  8.8484e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  5.4203e-06     	|  1              	|  5.4203e-06     	|  8.7507e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.99e-06       	|  1              	|  4.99e-06       	|  8.056e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.1951e-06     	|  2              	|  4.3903e-06     	|  7.0877e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.3605e-06     	|  1              	|  4.3605e-06     	|  7.0396e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  4.3204e-06     	|  1              	|  4.3204e-06     	|  6.975e-07      	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.2804e-06     	|  1              	|  4.2804e-06     	|  6.9103e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.1304e-06     	|  2              	|  4.2608e-06     	|  6.8788e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.1407e-06     	|  1              	|  4.1407e-06     	|  6.6848e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.054e-06      	|  2              	|  4.1081e-06     	|  6.6322e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  2.0401e-06     	|  2              	|  4.0801e-06     	|  6.5871e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  4.0503e-06     	|  1              	|  4.0503e-06     	|  6.539e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  3.9302e-06     	|  1              	|  3.9302e-06     	|  6.345e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.7104e-06     	|  1              	|  3.7104e-06     	|  5.9902e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.4105e-06     	|  1              	|  3.4105e-06     	|  5.506e-07      	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.2987e-06     	|  1              	|  3.2987e-06     	|  5.3256e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.6103e-06     	|  2              	|  3.2205e-06     	|  5.1993e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.7604e-06     	|  1              	|  2.7604e-06     	|  4.4565e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.37e-06       	|  2              	|  2.74e-06       	|  4.4235e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.6794e-06     	|  1              	|  2.6794e-06     	|  4.3257e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.5993e-06     	|  1              	|  2.5993e-06     	|  4.1964e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.3702e-06     	|  1              	|  2.3702e-06     	|  3.8265e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.36e-06       	|  1              	|  2.36e-06       	|  3.81e-07       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.3302e-06     	|  1              	|  2.3302e-06     	|  3.7619e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.3097e-06     	|  1              	|  2.3097e-06     	|  3.7288e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.3004e-06     	|  1              	|  2.3004e-06     	|  3.7138e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1998e-06     	|  1              	|  2.1998e-06     	|  3.5514e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.1802e-06     	|  1              	|  2.1802e-06     	|  3.5198e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.0796e-06     	|  1              	|  2.0796e-06     	|  3.3574e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.0396e-06     	|  1              	|  2.0396e-06     	|  3.2928e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.0303e-06     	|  1              	|  2.0303e-06     	|  3.2777e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.02e-06       	|  1              	|  2.02e-06       	|  3.2612e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  2.0005e-06     	|  1              	|  2.0005e-06     	|  3.2296e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.9502e-06     	|  1              	|  1.9502e-06     	|  3.1484e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  1.9399e-06     	|  1              	|  1.9399e-06     	|  3.1319e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.56e-06       	|  1              	|  1.56e-06       	|  2.5185e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  1.5404e-06     	|  1              	|  1.5404e-06     	|  2.4869e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.5199e-06     	|  1              	|  1.5199e-06     	|  2.4538e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.4901e-06     	|  1              	|  1.4901e-06     	|  2.4057e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1902e-06     	|  1              	|  1.1902e-06     	|  1.9215e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0701e-06     	|  1              	|  1.0701e-06     	|  1.7276e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  8.9966e-07     	|  1              	|  8.9966e-07     	|  1.4524e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
