.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/Custom/neural_net.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_Custom_neural_net.py: Simple Neural Network ===================== This example solves a simple binary classification problem using a basic neural network with 2 layers. The classification problem is generated by the make_moons dataset generator from scikit--learn. .. GENERATED FROM PYTHON SOURCE LINES 13-25 .. code-block:: Python import matplotlib.pyplot as plt from utils.plotting import plot_decision_boundary from tqdm import tqdm import torch as pt import torch.nn as nn from torch.utils.data import TensorDataset, DataLoader from sklearn.model_selection import train_test_split from skwdro.torch import robustify .. GENERATED FROM PYTHON SOURCE LINES 26-28 Problem setup ~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 28-38 .. code-block:: Python from sklearn.datasets import make_moons n = 512 + 64 X, y = make_moons(n_samples=n, noise=0.05, random_state=42) .. GENERATED FROM PYTHON SOURCE LINES 39-41 Visualize the data ~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 41-45 .. code-block:: Python plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.RdYlBu) # type: ignore plt.show() .. image-sg:: /examples/Custom/images/sphx_glr_neural_net_001.png :alt: neural net :srcset: /examples/Custom/images/sphx_glr_neural_net_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 46-48 Preprocessing ~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 48-75 .. code-block:: Python # Split the data into train and test sets X_train, X_test, y_train, y_test = train_test_split( X, y, train_size=512, test_size=64, random_state=42 ) device = "cuda" if pt.cuda.is_available() else "cpu" # Turn data into tensors full_batch_x = pt.from_numpy(X_train).to(device) full_batch_y = pt.from_numpy(y_train).unsqueeze(-1).to(full_batch_x) dataset = DataLoader( TensorDataset( full_batch_x, full_batch_y ), batch_size=64 ) batch_x_test = pt.from_numpy(X_train).to(device) batch_y_test = pt.from_numpy(y_train).unsqueeze(-1).to(batch_x_test) .. GENERATED FROM PYTHON SOURCE LINES 76-78 Two-layers model ~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 78-96 .. code-block:: Python class SimpleNN(nn.Module): def __init__(self, in_features, out_features, hidden_units): super().__init__() # Two hidden layers and logit output self.linear_relu_stack = nn.Sequential( nn.Linear(in_features, hidden_units), nn.ReLU(), nn.Linear(hidden_units, hidden_units), nn.ReLU(), nn.Linear(hidden_units, out_features), ) def forward(self, x): logits = self.linear_relu_stack(x) return logits .. GENERATED FROM PYTHON SOURCE LINES 97-99 Set the model up ~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 99-126 .. code-block:: Python model = SimpleNN( in_features=2, out_features=1, hidden_units=5 ).to(full_batch_x) print(model) loss_fn = nn.BCEWithLogitsLoss(reduction='none') # Define a sample batch for initialization sample_batch_x, sample_batch_y = next(iter(dataset)) # Robust loss robust_loss = robustify( loss_fn, model, pt.tensor(1e-4), sample_batch_x, sample_batch_y, cost_spec="t-NC-2-2", n_samples=16 ) # Replaces the loss of the model by the dual WDRO loss .. rst-class:: sphx-glr-script-out .. code-block:: none SimpleNN( (linear_relu_stack): Sequential( (0): Linear(in_features=2, out_features=5, bias=True) (1): ReLU() (2): Linear(in_features=5, out_features=5, bias=True) (3): ReLU() (4): Linear(in_features=5, out_features=1, bias=True) ) ) .. GENERATED FROM PYTHON SOURCE LINES 127-129 Training loop ~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 129-171 .. code-block:: Python pt.manual_seed(42) epochs = 250 # optimizer = pt.optim.AdamW(params=model.parameters(),lr=1e-2) optimizer = pt.optim.AdamW(params=robust_loss.parameters()) # Training loop iterator = tqdm(range(epochs), position=0, desc='Epochs', leave=False) losses = [] for epoch in iterator: avg_testloss = 0. for batch_x, batch_y in tqdm(dataset, position=1, desc='Sample', leave=False): # ## Training model.train() optimizer.zero_grad() # loss = loss_fn(model(batch_x.squeeze()), batch_y) loss = robust_loss(batch_x, batch_y, reset_sampler=True) loss.backward() optimizer.step() # ## Testing model.eval() with pt.no_grad(): # Forward pass test_logits = model(batch_x_test) test_pred = pt.round(pt.sigmoid(test_logits)) # Compute the loss avg_testloss += loss_fn(test_logits, batch_y_test).mean().item() iterator.set_postfix( {'acc': f"{(test_pred == batch_y_test).float().mean().item()*100}%"} ) losses.append(loss.item()) # Print iterator.set_postfix({'loss': avg_testloss / len(dataset)}) .. rst-class:: sphx-glr-script-out .. code-block:: none Epochs: 0%| | 0/250 [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: neural_net.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: neural_net.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_