Gradient Descent

NonArchimedeanMachineLearning.GradientDescentConfig — Type
GradientDescentConfig(; strict=false, degree=1, start_branch=1)

Configuration for gradient descent optimization.

Fields

  • strict::Bool: If true, descend one coordinate at a time; if false, descend all coordinates
  • degree::Int: Number of children to explore per polydisc node (non-strict mode)
  • start_branch::Int: Coordinate to descend along first (strict mode)

Example

config = GradientDescentConfig(strict = false, degree = 1)
optim = gradient_descent_init(param, loss, config)
source
NonArchimedeanMachineLearning.gradient_descent — Method
gradient_descent(loss::Loss, param::ValuationPolydisc{S,T,N}, next_branch::Int, settings::GradientDescentConfig) where {S,T,N}

Perform one step of gradient descent optimization.

Computes children of the current parameter point and selects the child that maximizes the gradient norm (steepest descent direction).

Arguments

  • loss::Loss: The loss function structure
  • param::ValuationPolydisc{S,T,N}: Current parameter values
  • next_branch::Int: Index of the next branch to descend in strict mode
  • settings::GradientDescentConfig: Configuration for gradient descent

Returns

Tuple{ValuationPolydisc{S,T,N}, Int, Bool}: New parameters, next branch index, and convergence status

source
NonArchimedeanMachineLearning.gradient_descent_init — Method
gradient_descent_init(param::ValuationPolydisc{S,T,N}, loss::Loss, settings::GradientDescentConfig=GradientDescentConfig()) where {S,T,N}

Initialize an optimization setup for gradient descent.

Arguments

  • param::ValuationPolydisc{S,T,N}: Initial parameter values
  • loss::Loss: The loss function structure
  • settings::GradientDescentConfig: Configuration controlling descent behavior

Returns

OptimSetup: Configured optimization setup for gradient descent

Notes

settings.start_branch is used only when strict mode is enabled.

source
NonArchimedeanMachineLearning.gradient_param — Method
gradient_param(m::AbstractModel, fun_eval::PolydiscFunctionEvaluator, val::ValuationPolydisc, v::ValuationTangent)

Compute the gradient of a model with respect to its parameters using a typed evaluator.

Arguments

  • m::AbstractModel: The abstract model (used for dimension info)
  • fun_eval::PolydiscFunctionEvaluator: Typed evaluator for the model function
  • val::ValuationPolydisc: Data variable values
  • v::ValuationTangent: Tangent vector in parameter space

Returns

Gradient vector with respect to parameters

source
NonArchimedeanMachineLearning.gradient_param — Method
gradient_param(m::AbstractModel{S}, val::ValuationPolydisc{S,T,N1}, v::ValuationTangent{S,T,N2}) where {S,T,N1,N2}

Compute the gradient of a model with respect to its parameters using the symbolic path.

Arguments

  • m::AbstractModel{S}: The abstract model
  • val::ValuationPolydisc{S,T,N1}: Data variable values
  • v::ValuationTangent{S,T,N2}: Tangent vector in parameter space

Returns

Gradient vector with respect to parameters

Notes

Currently assumes parameters are the last variables. More general shapes may be needed.

source