AI 模型是不是越大越聰明？

很多人直覺上會覺得：AI 模型越大，參數越多，就一定越聰明。

這個判斷只對了一半。模型規模確實很重要，特別是在預訓練階段，更多參數、更多資料、更多算力通常能帶來更好的基礎能力。但到了今天，模型是否「聰明」已經不只看大小，還要看資料品質、架構設計、推理方法、工具使用能力，以及它被優化的任務場景。

換句話說，大模型不一定永遠贏；小模型也不一定只能做簡單任務。

為什麼「越大越聰明」不再夠用？

早期大型語言模型的發展很依賴 scaling laws：模型越大、資料越多、訓練算力越高，整體表現往往越好。這仍然是基礎模型發展的重要規律，但它不是唯一規律。

現在判斷一個模型強不強，至少要看下面幾件事。

1. 資料品質比資料數量更關鍵

一個 7B 參數的小模型，如果使用高品質、乾淨、結構良好的資料訓練，可能在特定任務上超過一個資料品質較差的 70B 模型。

原因很直觀：模型不是單純「吃越多文字越好」，它會學到資料中的推理方式、語言習慣、錯誤模式和偏見。如果訓練資料裡有大量重複內容、低品質網頁、錯誤答案或機器生成垃圾，模型也會把這些缺陷吸收進去。

所以近年的模型訓練越來越重視：

  • 資料清洗與去重
  • 高品質教科書、論文、程式碼與解題資料
  • 合成資料與人工驗證資料
  • 從強模型蒸餾出的高品質推理樣本

2. 架構會影響「有效參數」

有些模型採用 MoE（Mixture of Experts，混合專家）架構。MoE 模型可以擁有非常多的總參數，但每次推理時只啟用其中一部分專家。

這代表兩件事：

  • 模型的「總參數」不等於每次回答實際動用的「活躍參數」。
  • 一個模型可以在儲存上很大，但推理成本不一定等比例增加。

例如 DeepSeek-V3 / R1 系列公開資料顯示，它們是 671B 總參數的 MoE 模型，但每個 token 只啟用約 37B 參數。這種設計讓模型能兼顧容量與推理效率。

3. 推理能力和知識量不是同一件事

大模型通常有更強的知識覆蓋能力。它像一本更大的百科全書，能記住更多冷門事實、語言模式和跨領域資訊。

但推理模型的進步改變了另一件事：模型可以用更多「思考時間」換取更好的解題能力。像 OpenAI o 系列、DeepSeek-R1 這類 reasoning model，重點不只是把答案背出來，而是在回答前產生更長的中間推理、檢查、修正和嘗試。

這讓一些較小的模型在數學、程式碼、邏輯題等任務上，能接近甚至超過過去更大的通用模型。

4. 小模型在邊緣裝置上越來越實用

手機、筆記型電腦和一般工作站已經可以執行 3B、7B、14B 甚至更大的量化模型。這些模型不一定適合處理所有問題，但在摘要、改寫、分類、簡單問答、程式輔助、個人知識庫檢索等任務上，已經很有實用價值。

小模型的優勢包括：

  • 成本低
  • 延遲低
  • 可離線使用
  • 隱私更好
  • 可以針對特定任務微調

它們的短板則是：

  • 冷門知識較弱
  • 長上下文能力通常較弱
  • 複雜任務穩定性不如前沿大模型
  • 容易在知識不足時產生幻覺

Claude、GPT 這類前沿模型到底有多大？

OpenAI、Anthropic 等公司通常不公開前沿閉源模型的參數量、架構細節、訓練資料和硬體配置。外界有很多推估，例如從 API 價格、推理延遲、晶片成本、論文線索或模型行為反推，但這些都不是官方數字。

因此，像「某某模型有 9.7T 參數」、「某某 Claude 有 5T 參數」這種說法，要非常小心看待。它們可以作為市場傳聞或研究推測，但不適合寫成確定事實。

比較穩健的說法是：

  • 前沿閉源模型的具體參數量通常未公開。
  • 它們可能使用非常大的 dense 或 MoE 架構，也可能結合多模型路由、工具使用、檢索系統和測試時計算。
  • 真正影響使用者體驗的，不只是參數量，還包括後訓練、對齊、推理策略、上下文長度、工具調用、系統工程和服務端路由。

模型檔案大小怎麼估算？

如果只粗略估算模型權重大小，可以用下面的方式理解：

  • FP16 / BF16：每個參數約 2 bytes
  • INT8：每個參數約 1 byte
  • INT4：每個參數約 0.5 byte

所以一個 7B 模型：

  • FP16 約 14GB
  • INT8 約 7GB
  • INT4 約 3.5GB 到 5GB 左右，實際大小還會受格式、metadata、量化方法影響

一個 14B 模型：

  • FP16 約 28GB
  • INT8 約 14GB
  • INT4 約 7GB 到 9GB 左右

這也是為什麼「10GB 以下」通常對應到 7B 或 14B 等級的量化模型，而不是完整未壓縮的大模型。

有沒有可能用 10GB 以下做出接近 Claude Opus 的模型？

要看「接近」指的是什麼。

如果指的是所有能力都等同於 Claude Opus 這類前沿閉源模型，那目前不現實。10GB 以下的小模型無法同時擁有同等級的百科知識、長上下文穩定性、多語能力、工具整合、複雜任務泛化能力和安全對齊。

但如果只看特定任務，例如數學解題、程式碼補全、格式化輸出、客服分類、文件摘要、固定領域問答，那 10GB 以下的小模型完全可能接近甚至超過大型通用模型。

關鍵在於三個技術方向。

1. 用思考時間換模型體積

推理模型會生成較長的中間推理，透過嘗試、檢查和修正提升答案品質。這有點像讓一個較小的模型多花一點時間做題，而不是要求它立刻反射式回答。

這種方法在數學和程式任務上特別有效，因為答案可以被驗證：數學題有標準答案，程式碼可以跑測試，邏輯題可以檢查約束。

2. 用知識蒸餾濃縮能力

知識蒸餾的概念是讓強模型當老師，產生高品質解題過程、範例答案或任務資料，再讓小模型學習。

小模型不需要重新讀完整個網路，也不需要自己摸索所有推理路徑。它可以直接學習強模型留下的解題痕跡。DeepSeek-R1 發布時，就同時開源了多個從 R1 蒸餾而來的小型 dense models，包含 1.5B、7B、8B、14B、32B、70B 等版本。

3. 用量化降低硬體需求

量化可以把模型權重從 FP16 壓到 INT8、INT4，讓模型更容易放進一般顯示卡或 Mac 的統一記憶體裡。

不過量化不是免費午餐。壓得越狠，模型越可能出現精度下降、推理不穩、長上下文退化或特定任務能力受損。好的量化方法可以保留大部分能力，但不能保證所有模型、所有任務都維持同樣表現。

10GB 小模型適合什麼？

適合：

  • 個人離線助理
  • 文件摘要與改寫
  • 程式碼解釋與簡單除錯
  • 固定領域客服
  • 搭配 RAG 的知識庫問答
  • 資料抽取與格式化
  • 隱私敏感的本地工作流

不太適合：

  • 需要大量冷門知識的開放問答
  • 超長文件的精準跨段推理
  • 高風險醫療、法律、金融決策
  • 需要穩定多步工具調用的複雜代理任務
  • 對錯誤容忍度很低的生產決策

結論

AI 模型不是越大就一定越聰明。更準確的說法是：規模提供上限，但資料、架構、後訓練、推理策略和任務設計，決定模型能不能把這個上限發揮出來。

大模型像一個知識面極廣、泛化能力強的專家團隊；小模型則像一個被訓練得很專精、反應很快、成本很低的助手。

如果你要的是通用能力、長文本理解、跨領域穩定性，前沿大模型仍然有明顯優勢。
如果你要的是本地部署、低成本、隱私、固定任務和快速反應，小模型已經非常值得認真考慮。

參考資料

Resolving VSCode Crash After Update on macOS

Recently, I encountered a frustrating issue with Visual Studio Code (VSCode) on my macOS system. After performing an update, the application refused to open, effectively crashing on startup. If you’re facing a similar problem, you’re not alone, and there’s a solution available.

The Issue

The problem appears to be affecting macOS users who recently updated their VSCode installation. Upon attempting to launch the application, it fails to open, leaving users unable to access their development environment.
The problem I experienced is exactly the same as described in this Stack Overflow question:
https://stackoverflow.com/questions/78710474/cannot-open-vscode-on-macos-after-update

The Solution

After some research, I found that this issue has been reported and addressed by the VSCode team. Here’s how you can resolve it:

  1. Download VSCode version 1.90
    The key to fixing this problem is to revert to a previous, stable version of VSCode. Specifically, you’ll need to download and install VSCode version 1.90.

    You can find the download link for VSCode 1.90 on the official Visual Studio Code updates page:
    VSCode 1.90 Download

  2. Install the downloaded version
    Once you’ve downloaded VSCode 1.90, install it on your macOS system. This should replace the problematic updated version.

  3. Launch VSCode
    After installation, try launching VSCode again. It should now open without crashing.

Additional Information

For those interested in the technical details or ongoing discussions about this issue, you can refer to the following GitHub issue threads:

These threads contain valuable information about the nature of the problem and potential long-term fixes that the VSCode team may implement.

Conclusion

While it’s inconvenient to encounter such issues after an update, it’s reassuring to know that the VSCode community and development team are quick to identify and provide solutions. By reverting to version 1.90, you should be able to continue using VSCode without interruption.

Remember to keep an eye on future updates, as the VSCode team will likely address this issue in upcoming releases.

How to Use Git Tag

Git tags are used to mark specific points in your repository’s history. This is typically done to mark release points (e.g., v1.0, v2.0). There are two types of tags in Git: lightweight and annotated. Here’s how to use them:

Creating Tags

  1. Lightweight Tag:
    A lightweight tag is like a branch that doesn’t change. It’s just a pointer to a commit.

    1
    git tag <tagname>

    Example:

    1
    git tag v1.0

  2. Annotated Tag:
    Annotated tags are stored as full objects in the Git database. They can have a message, and metadata such as the tagger’s name, email, and date.

    1
    git tag -a <tagname> -m "message"

    Example:

    1
    git tag -a v1.0 -m "Release version 1.0"

Viewing Tags

To list all tags in your repository:

1
git tag

To view details of a specific tag:

1
git show <tagname>

Example:

1
git show v1.0

Sharing Tags

By default, git push does not push tags to remote repositories. To push a specific tag:

1
git push origin <tagname>

Example:

1
git push origin v1.0

To push all tags at once:

1
git push origin --tags

Deleting Tags

To delete a local tag:

1
git tag -d <tagname>

Example:

1
git tag -d v1.0

To delete a remote tag:

1
git push origin --delete <tagname>

Example:

1
git push origin --delete v1.0

Checking Out Tags

To checkout a specific tag (this puts you in a “detached HEAD” state, meaning you are not on a branch):

1
git checkout <tagname>

Example:

1
git checkout v1.0

Remember that when you are in a detached HEAD state, any changes you make are not associated with any branch unless you create a new branch from this state.

Summary

  • Create a lightweight tag: git tag <tagname>
  • Create an annotated tag: git tag -a <tagname> -m "message"
  • List tags: git tag
  • Show tag details: git show <tagname>
  • Push a specific tag: git push origin <tagname>
  • Push all tags: git push origin --tags
  • Delete a local tag: git tag -d <tagname>
  • Delete a remote tag: git push origin --delete <tagname>
  • Checkout a tag: git checkout <tagname>

Using these commands, you can effectively manage tags in your Git repositories.

Best Practices for Managing Side Effects in React with `useEffect`

When working with useEffect in React, the way you handle subscriptions is crucial for maintaining performance and avoiding memory leaks. Let’s explore two forms of useEffect for managing authentication state changes using Firebase’s onAuthStateChanged and understand why one approach is superior.

Handling Subscriptions with Cleanup

First, consider the following example where we properly manage the subscription by including an unsubscribe mechanism:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
useEffect(() => {
  const auth = getAuth();
  const unsubscribe = onAuthStateChanged(auth, (user) => {
    if (user) {
      const userProfile: UserInfo = user.providerData[0];
      setUserInfo({
        providerId: userProfile.providerId,
        uid: userProfile.uid,
        displayName: userProfile.displayName,
        email: userProfile.email,
        photoURL: userProfile.photoURL,
      });
    }
    setLoading(false);
  });

  return () => unsubscribe();
}, []);

Handling Subscriptions Without Cleanup

Now, compare it with a version that does not manage the subscription cleanup:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
useEffect(() => {
  const auth = getAuth();
  onAuthStateChanged(auth, (user) => {
    if (user) {
      const userProfile: UserInfo = user.providerData[0];
      setUserInfo({
        providerId: userProfile.providerId,
        uid: userProfile.uid,
        displayName: userProfile.displayName,
        email: userProfile.email,
        photoURL: userProfile.photoURL,
      });
    }
    setLoading(false);
  });
}, []);

Why the Unsubscribe Form is Preferable

Using the form with unsubscribe is generally better for several reasons:

  1. Memory Management:

    • Subscriptions persist until explicitly cancelled. By calling unsubscribe in the cleanup phase of useEffect, you ensure the subscription is terminated when the component unmounts or the effect dependencies change, preventing memory leaks.

  2. Avoiding Multiple Subscriptions:

    • If the dependencies of the useEffect change, the effect re-runs. Without cleanup, each re-run creates a new subscription, leading to multiple active subscriptions and potential issues. The unsubscribe method ensures only one active subscription at a time.

  3. Best Practices:

    • Explicitly handling side effect cleanups is a best practice in React. It ensures your component releases resources correctly, leading to predictable behavior and preventing issues caused by stale listeners or handlers.

Conclusion

While the second form (without unsubscribe) may be adequate for simple scenarios or quick experiments, the first form (with unsubscribe) is more robust. It aligns with React’s best practices for managing side effects and resource cleanup, ensuring your component remains performant and reliable, particularly in larger or more complex applications. Always favor the approach that includes cleanup to maintain optimal performance and avoid potential pitfalls.

retro warning message 

1
[W NNPACK.cpp:61] Could not initialize NNPACK! Reason: Unsupported hardware.

https://github.com/pytorch/pytorch/blob/70f4b3551c01230d4ab00da7bf453fa7c6b14eb9/aten/src/ATen/native/NNPACK.cpp#L52-L72

https://discuss.pytorch.org/t/bug-w-nnpack-cpp-80-could-not-initialize-nnpack-reason-unsupported-hardware/107518/23

Environment Architecture CPU requirements
Linux x86-64 AVX2 and 3-level cache hierarchy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
$ lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         36 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
    CPU family:          6
    Model:               42
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            7
    CPU max MHz:         3800.0000
    CPU min MHz:         1600.0000
    BogoMIPS:            6784.75
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mc
                         a cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht
                         tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon p
                         ebs bts rep_good nopl xtopology nonstop_tsc cpuid aperf
                         mperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est t
                         m2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcn
                         t tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd
                          ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid
                         xsaveopt dtherm ida arat pln pts md_clear flush_l1d
Virtualization features:
  Virtualisation:        VT-x
Caches (sum of all):
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    1 MiB (4 instances)
  L3:                    8 MiB (1 instance)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-7
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushe
                         s, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Unknown: No mitigations
  Retbleed:              Not affected
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
                          and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer
                          sanitization
  Spectre v2:            Mitigation; Retpolines; IBPB conditional; IBRS_FW; STIB
                         P conditional; RSB filling; PBRSB-eIBRS Not affected; B
                         HI Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

AVX2 (Advanced Vector Extensions 2) is an expansion of the AVX instruction set introduced by Intel and AMD. It provides additional instructions to accelerate integer vector operations. AVX2 builds upon the foundation of AVX, offering higher performance and efficiency for certain computational workloads.

AVX2 introduces several new features, including:

  • Support for 256-bit integer vector operations: AVX2 extends the width of vector registers from 128 bits to 256 bits, allowing for more data to be processed simultaneously.
  • New integer vector instructions: AVX2 adds new instructions for integer vector arithmetic, bitwise operations, and vector shuffles, providing - enhanced capabilities for tasks such as image processing, cryptography, and scientific computing.
  • Enhanced gather and scatter operations: AVX2 includes new instructions for indexed memory operations, enabling more efficient data movement - between memory and vector registers.

CPUs that support AVX2 typically belong to newer generations and include various Intel and AMD processors released after 2013. Some examples of CPUs that support AVX2 include:

  • Intel Haswell (4th generation Core processors) and newer.
  • Intel Broadwell, Skylake, Kaby Lake, Coffee Lake, Comet Lake, and later microarchitectures.
  • AMD Ryzen processors (starting with the first-generation Ryzen CPUs).
  • AMD Ryzen Threadripper processors.
  • AMD EPYC processors.

These processors offer improved performance for workloads that can leverage AVX2 instructions, making them well-suited for tasks involving heavy computational workloads, such as machine learning, numerical simulations, and media processing.

pyenv install notes

https://github.com/pyenv/pyenv

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ curl https://pyenv.run | bash

Cloning into '~/.pyenv'...


WARNING: seems you still have not added 'pyenv' to the load path.

# Load pyenv automatically by appending
# the following to
# ~/.bash_profile if it exists, otherwise ~/.profile (for login shells)
# and ~/.bashrc (for interactive shells) :

export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# Restart your shell for the changes to take effect.

# Load pyenv-virtualenv automatically by adding
# the following to ~/.bashrc:

eval "$(pyenv virtualenv-init -)"